On Default Priors for Robust Bayesian Estimation with Divergences

Abstract

This paper presents objective priors for robust Bayesian estimation against outliers based on divergences. The minimum $\gamma$-divergence estimator is well-known to work well in estimation against heavy contamination. The robust Bayesian methods by using quasi-posterior distributions based on divergences have been also proposed in recent years. In the objective Bayesian framework, the selection of default prior distributions under such quasi-posterior distributions is an important problem. In this study, we provide some properties of reference and moment matching priors under the quasi-posterior distribution based on the $\gamma$-divergence. In particular, we show that the proposed priors are approximately robust under the condition on the contamination distribution without assuming any conditions on the contamination ratio. Some simulation studies are also presented.

1. Introduction

The problem of the robust parameter estimation against outliers has a long history. For example, Huber and Ronchetti [1] provided an excellent review of the classical robust estimation theory. It is well-known that the maximum likelihood estimator (MLE) is not robust against outliers because it is obtained by minimizing the Kullback–Leibler (KL) divergence between the true and empirical distributions. To overcome this problem, we may use other (robust) divergences instead of the KL divergence. The robust parameter estimation based on divergences has been one of the central topics in modern robust statistics (e.g., [2]). Such a method was firstly proposed by [3], who referred to it as the minimum density power divergence estimator. Reference [4] also proposed the “type 0 divergence”, which is a modified version of the density power divergence, and Reference [5] showed that it has good robustness properties. The type 0 divergence is also known as the $\gamma$-divergence, and statistical methods based on the $\gamma$-divergence have been presented by many authors (e.g., [6,7,8]).

In Bayesian statistics, the robustness against outliers is also an important issue, and divergence-based Bayesian methods have been proposed in recent years. Such methods are known as quasi-Bayes (or general Bayes) methods in some studies, and the corresponding posterior distributions are called quasi-posterior (or general posterior) distributions. To overcome the model misspecification problem (see [9]), the quasi-posterior distributions are based on a general loss function rather than the usual log-likelihood function. In general, such general loss functions may not depend on an assumed statistical model. However, in this study, we use loss functions that depend on the assumed model because we are interested in the robust estimation problem against outliers, that is the model is not misspecified, but the data generating distribution is wrong. In other words, we use divergences or scoring rules as a loss function for the quasi-posterior distribution (see also [10,11,12,13,14]). For example, Reference [10] used the Hellinger divergence. Reference [11] used the density power divergence. References [12,14] used the $\gamma$-divergence. In particular, the quasi-posterior distribution based on the $\gamma$-divergence was referred to as the $\gamma$-posterior in [12], and they showed that the $\gamma$-posterior has good robustness properties to overcome the problems in [11].

Although the selection of priors is an important issue in Bayesian statistics, we often have no prior information in some practical situations. In such cases, we may use priors called default or objective priors, and we should select an appropriate objective prior in a given context. In particular, we consider the reference and moment matching priors in this paper. The reference prior was firstly proposed by [15], and the moment matching prior was proposed by [16]. However, such objective priors generally depend on an unknown data generating distribution when we cannot assume that the contamination ratio is approximately zero. For example, if we assume the $\epsilon$-contamination model (see, e.g., [1]) as a data generating distribution, many objective priors depend on the unknown contamination ratio and unknown contamination distribution because these objective priors involve the expectations under the data generating distribution. Although [17] derived some kinds of reference priors under the quasi-posterior distributions based on some kinds of scoring rules, they only discussed the robustness of such reference priors when the contamination ratio $\epsilon$ is approximately zero. Furthermore, their simulation studies largely depended on the assumption for the contamination ratio. In other words, they indirectly assumed that the contamination ratio $\epsilon$ is approximately zero. The current study derives the moment matching priors under the quasi-posterior distribution in a similar way as [16], and we show that the reference and moment matching priors based on the $\gamma$-divergence do not approximately depend on such unknown quantities under a certain assumption for the contamination distribution even if the contamination ratio is not small.

The rest of this paper is organized as follows. In Section 2, we review robust Bayesian estimation based on divergences referring to some previous studies. We derive moment matching priors based on the quasi-posterior distribution using an asymptotic expansion of the quasi-posterior distribution given by [17] in Section 3. Furthermore, we show that the reference and moment matching priors based on the $\gamma$-posterior do not depend on the contamination ratio and the contamination distribution. In Section 4, we compare the empirical bias and mean squared error of posterior means through some simulation studies. Some discussion about the selection of tuning parameters is also provided.

2. Robust Bayesian Estimation Using Divergences

In this section, we review the framework of robust estimation in the seminal paper by Fujisawa and Eguchi [5], and we introduce the robust Bayesian estimation using divergences. Let $X_1,\dots ,X_n$ be independent and identically distributed (iid) random variables according to a distribution G with the probability density function g on $\mathsf{\Omega }$, and let ${\mathit{X}}_n=(X_1,\cdots ,X_n)$. We assume the parametric model $f_{\mathit{\theta }}=f(x,\mathit{\theta })$ ($\mathit{\theta }\in \mathsf{\Theta }\subset {\mathbb{R}}^p$) and consider the estimation problem for $\mathit{\theta }$.

Then, the $\gamma$-divergence between two probability densities g and f is defined by:

$$\begin{array}{cc}\hfill D_{\gamma }(g,f_{\mathit{\theta }})=& \frac{1}{\gamma (\gamma +1)}log{\int }_{\mathsf{\Omega }}g{\left(x\right)}^{1+\gamma }\mathrm{d}x\hfill \\ \hfill & -\frac{1}{\gamma }log{\int }_{\mathsf{\Omega }}g\left(x\right)f_{\mathit{\theta }}{\left(x\right)}^{\gamma }\mathrm{d}x+\frac{1}{\gamma +1}log{\int }_{\mathsf{\Omega }}{f}_{\mathit{\theta }}{\left(x\right)}^{1+\gamma }\mathrm{d}x,\hfill \end{array}$$

where $\gamma >0$ is a tuning parameter on robustness. We also define the $\gamma$-cross-entropy as:

$$d_{\gamma }(g,f_{\mathit{\theta }})=-\frac{1}{\gamma }log{\int }_{\mathsf{\Omega }}g\left(x\right)f_{\mathit{\theta }}{\left(x\right)}^{\gamma }\mathrm{d}x+\frac{1}{\gamma +1}log{\int }_{\mathsf{\Omega }}{f}_{\mathit{\theta }}{\left(x\right)}^{1+\gamma }\mathrm{d}x$$

(see [4,5]).

2.1. Framework of Robustness

Fujisawa and Eguchi [5] introduced a new framework of robustness, which is different from the classical one. When some of the data values are regarded as outliers, we need a robust estimation procedure. Typically, an observation that takes a large value is regarded as an outlier. Under this convention, many robust parameter estimation procedures have been proposed to reduce the bias caused by an outlier. An influence function is one of the methods to measure the sensitivity of models against outliers. It is known that the bias of an estimator is approximately proportional to the influence function when the contamination ratio $\epsilon$ is small. However, when $\epsilon$ is not small, the bias cannot be approximately proportional to the influence function. Reference [5] showed that the likelihood function based on the $\gamma$-divergence gives a sufficiently small bias under heavy contamination. Suppose that observations are generated from a mixture distribution $g\left(x\right)=(1-\epsilon )f\left(x\right)+\epsilon \delta \left(x\right)$, where $f\left(x\right)$ is the underlying density, $\delta \left(x\right)$ is another density function, and $\epsilon$ is the contamination ratio. In Section 3, we assume that the condition:

$$\begin{array}{c}\hfill {\nu }_f={\left\{{\int }_{\mathsf{\Omega }}\delta \left(x\right)f{\left(x\right)}^{{\gamma }_0}\mathrm{d}x\right\}}^{1/{\gamma }_0}\approx 0\end{array}$$

(1)

holds for a constant ${\gamma }_0>0$ (see [5]). When $x_0$ is generated from $\delta \left(x\right)$, we call $x_0$ the outlier. We note that we do not assume that the contamination ratio $\epsilon$ is sufficiently small. This condition means that the contamination distribution $\delta \left(x\right)$ mostly lies on the tail of the underlying density $f\left(x\right)$. In other words, for an outlier $x_0$, it holds that $f\left(x_0\right)\approx 0$. We note that the condition (1) is also a basis to prove the robustness against outliers for the minimum $\gamma$-divergence estimator in [5]. Furthermore, Reference [18] provided some theoretical results for the $\gamma$-divergence, and related works in the frequentist setting have been also developed (e.g., [6,7,8], and so on).

In the rest of this section, we give a brief review of the general Bayesian updating and introduce some previous works that are closely related to this paper.

2.2. General Bayesian Updating

We consider the same framework as [9,13]. We are interested in $\mathit{\theta }=\mathit{\theta }\left(G\right)$ ($\mathit{\theta }\in \mathsf{\Theta }\subseteq {\mathbb{R}}^p$), and we define a loss function ${\ell }_{\mathit{\theta }}\left(\mathit{x}\right):=\ell (\mathit{\theta },\mathit{x})$. Further, let ${\mathit{\theta }}^{*}=arg{min}_{\mathit{\theta }\in \mathsf{\Theta }}{\mathbb{E}}_G{\ell }_{\mathit{\theta }}\left(\mathit{X}\right)$ be the target parameter. We define the risk function by ${\mathbb{E}}_G{\ell }_{\mathit{\theta }}\left(\mathit{X}\right)$, and its empirical risk is defined by $R_n\left(\mathit{\theta }\right)=(1/n){\sum }_{i=1}^n{\ell }_{\mathit{\theta }}\left(X_i\right)$. For the prior distribution $\pi \left(\theta \right)$, the quasi-posterior density is defined by:

$$\begin{array}{c}\hfill {\pi }_{n,\omega }\left(\mathit{\theta }\right)\propto exp\{-\omega n{R}_n\left(\mathit{\theta }\right)\}\pi \left(\mathit{\theta }\right),\end{array}$$

where $\omega >0$ is a tuning parameter called the learning rate. We note that the quasi-posterior is also called the general posterior or Gibbs posterior. In this paper, we fix $\omega =1$ for the same reason as [13]. For example, if we set ${\ell }_{\mu }\left(x\right)=|x-\mu |$, we can estimate the median of the distribution without assuming the statistical model. However, we consider the model-dependent loss function, which is based on statistical divergence (or the scoring rule) in this study (see also [11,12,13,14]). The unified framework of inference using the quasi-posterior distribution was discussed by [9].

2.3. Assumptions and Previous Works

Let $d(·,·)$ be a cross-entropy induced by a divergence, and let $\{f_{\mathit{\theta }}:\mathit{\theta }\in \mathsf{\Theta }\}$ be a statistical model. In general, the quasi-posterior distribution based on the cross-entropy is defined by:

$$\begin{array}{c}\hfill \pi {}^{\left(d\right)}\left(\mathit{\theta }\right|{\mathit{X}}_n)\propto exp\left\{-nd(\overline{g},f_{\mathit{\theta }})\right\}\pi \left(\mathit{\theta }\right)=exp\left\{\sum _{i=1}^n{q}^{\left(d\right)}(X_i;\mathit{\theta })\right\}\pi \left(\mathit{\theta }\right),\end{array}$$

(2)

where $d(\overline{g},f_{\mathit{\theta }})$ is the empirically estimated cross-entropy and $\overline{g}$ is the empirical density function. In robust statistics based on divergences, we may use the cross-entropy induced by a robust divergence (e.g., [3,4,5]). In this paper, we mainly use the $\gamma$-cross-entropy proposed by [4,5]. Recently, Reference [12] proposed the $\gamma$-posterior based on the monotone transformation of the $\gamma$-cross-entropy:

$$\begin{array}{cc}\hfill {\tilde{d}}_{\gamma }(g,f_{\mathit{\theta }})& =-\frac{1}{\gamma }\left\{exp(-\gamma d_{\gamma }(g,f_{\mathit{\theta }}))-1\right\}=-\frac{1}{\gamma }\frac{{\int }_{\mathsf{\Omega }}g\left(x\right)f_{\mathit{\theta }}{\left(x\right)}^{\gamma }\mathrm{d}x}{{\left({\int }_{\mathsf{\Omega }}{f}_{\mathit{\theta }}{\left(x\right)}^{1+\gamma }\mathrm{d}x\right)}^{\gamma /(1+\gamma )}}+\frac{1}{\gamma }\hfill \end{array}$$

for $\gamma >0$. The $\gamma$-posterior is defined by taking $d(\overline{g},f_{\mathit{\theta }})={\tilde{d}}_{\gamma }(\overline{g},f_{\mathit{\theta }})$ in (2). On the other hand, Reference [11] proposed the $R^{\left(\alpha \right)}$-posterior based on the density power cross-entropy:

$$d_{\alpha }(g,f_{\mathit{\theta }})=-\frac{1}{\alpha }\int g{f}_{\mathit{\theta }}^{\alpha }\mathrm{d}x+\frac{1}{1+\alpha }log{\int }_{\mathsf{\Omega }}{f}_{\mathit{\theta }}^{1+\alpha }\mathrm{d}x$$

for $\alpha >0$. The $R^{\left(\alpha \right)}$-posterior is defined by taking $d(\overline{g},f_{\theta })=d_{\alpha }(\overline{g},f_{\theta })$ in (2). Note that cross-entropies $d_{\alpha }(·,·)$ and ${\tilde{d}}_{\gamma }(·,·)$ converge to the negative log-likelihood function as $\alpha \to 0$ and $\gamma \to 0$, respectively. Hence, we can establish that they are some kind of generalization of the negative log-likelihood function. It is known that the posterior mean based on the $R^{\left(\alpha \right)}$-posterior works well for the estimation of a location parameter in the presence of outliers. However, this is known to be unstable in the case of the estimation for a scale parameter (see [12]). Nakagawa and Hashimoto [12] showed that the posterior mean under the $\gamma$-posterior has a small bias under heavy contamination for both location and scale parameters in some simulation studies.

Let ${\mathit{\theta }}_g:=arg{min}_{\mathit{\theta }\in \mathsf{\Theta }}d(g,f_{\mathit{\theta }})$ be the target parameter. We now assume the following regularity conditions on the density function $f_{\mathit{\theta }}\left(x\right)=f(x;\mathit{\theta })(\mathit{\theta }\in \mathsf{\Theta }\subset {\mathbb{R}}^p)$. We use indices to denote derivatives of $\overline{D}\left(\mathit{\theta }\right)=d(\overline{g},f_{\mathit{\theta }})$ with respect to the components of the parameter $\mathit{\theta }$. For example, ${\overline{D}}_{ijk}\left(\mathit{\theta }\right)={\partial }_i{\partial }_j{\partial }_k\overline{D}\left(\mathit{\theta }\right)$ and ${\overline{D}}_{ijk\ell }\left(\mathit{\theta }\right)={\partial }_i{\partial }_j{\partial }_k{\partial }_{\ell }\overline{D}\left(\mathit{\theta }\right)$ for $i,j,k,\ell =1,\dots ,p$.

(A1)

The support of the density function does not depend on unknown parameter $\mathit{\theta }$, and $f_{\mathit{\theta }}$ is fifth-order differentiable with respect to $\mathit{\theta }$ in neighborhood U of ${\mathit{\theta }}_g$.

(A2)

The interchange of the order of integration with respect to x and differentiation as ${\mathit{\theta }}_g$ is justified. The expectations:

$${\mathbb{E}}_g\left[{\partial }_i{\partial }_j{\partial }_k{q}^{\left(d\right)}(X_1;{\mathit{\theta }}_g)\right]\mathrm{and}{\mathbb{E}}_g\left[{\partial }_i{\partial }_j{\partial }_k{\partial }_{\ell }{q}^{\left(d\right)}(X_1;{\mathit{\theta }}_g)\right]$$

are all finite, and $M_{ijk\ell s}\left(x\right)$ exists such that:

$$\begin{array}{c}\hfill \underset{\mathit{\theta }\in U}{sup}\left|{\partial }_i{\partial }_j{\partial }_k{\partial }_{\ell }{\partial }_s{q}^{\left(d\right)}(x;\mathit{\theta })\right|\le M_{ijk\ell s}\left(x\right)\end{array}$$

and ${\mathbb{E}}_g\left[M_{ijk\ell s}\left(X_1\right)\right]<\infty$ for all $i,j,k,\ell ,s=1,\dots ,p$, where ${\partial }_i=\partial /\partial {\theta }_i$ and $\partial =\partial /\partial \mathit{\theta }$, while ${\mathbb{E}}_g(·)$ is the expectation of X with respect to a probability density function g.

(A3)

For any $\delta >0$, with probability one:

$$\begin{array}{c}\hfill \underset{\parallel \mathit{\theta }-{\mathit{\theta }}_g\parallel >\delta }{sup}\left\{d(\overline{g},f_{{\mathit{\theta }}_g})-d(\overline{g},f_{\mathit{\theta }})\right\}<-\epsilon \end{array}$$

for some $\epsilon >0$ and for all sufficiently large n.

The matrices $I^{\left(d\right)}\left(\mathit{\theta }\right)$ and $J^{\left(d\right)}\left(\mathit{\theta }\right)$ are defined by:

$$\begin{array}{cc}\hfill & I^{\left(d\right)}\left(\mathit{\theta }\right)={\mathbb{E}}_g\left[\partial q^{\left(d\right)}(X_1;\mathit{\theta }){\partial }^{\top }{q}^{\left(d\right)}(X_1;\mathit{\theta })\right],\hfill \\ \hfill & J^{\left(d\right)}\left(\mathit{\theta }\right)=-{\mathbb{E}}_g\left[\partial {\partial }^{\top }{q}^{\left(d\right)}(X_1;\mathit{\theta })\right],\hfill \end{array}$$

respectively. We also assume that $I^{\left(d\right)}\left(\mathit{\theta }\right)$ and $J^{\left(d\right)}\left(\mathit{\theta }\right)$ are positive definite matrices. Under these conditions, References [11,12] discussed several asymptotic properties of the quasi-posterior distributions and the corresponding posterior means.

In terms of the higher order asymptotic theory, Giummolè et al. [17] derived the asymptotic expansion of such quasi-posterior distributions. We now introduce the notation that will be used in the rest of the paper. Reference [17] presented the following theorem.

Theorem 1 

(Giummolè et al. [17]). Under the conditions (A1)–(A3), we assume that ${\hat{\mathit{\theta }}}_n^{\left(d\right)}$ is a consistent solution of $\partial d(\overline{g},f_{\mathit{\theta }})=\mathbf{0}$ and ${\hat{\mathit{\theta }}}_n^{\left(d\right)}\stackrel{p}{\to }{\mathit{\theta }}_g$ as $n\to \infty$. Then, for any prior density function $\pi \left(\mathit{\theta }\right)$ that is third-order differentiable and positive at ${\mathit{\theta }}_g$, it holds that:

$${\pi }^{*\left(d\right)}\left(t_n\right|{\mathit{X}}_n)=\varphi \left(t_n;{\tilde{J}}^{-1}\right)\left(1+n^{-1/2}{A}_1\left(t_n\right)+n^{-1}{A}_2\left(t_n\right)\right)+O_p\left(n^{-3/2}\right),$$

(3)

where ${\pi }^{*\left(d\right)}\left(t_n\right|{\mathit{X}}_n)$ is the quasi-posterior density function of the normalized random variable $t_n={(t_1,\dots ,t_p)}^{\top }=\sqrt{n}(\mathit{\theta }-{\hat{\mathit{\theta }}}_n^{\left(d\right)})$ given ${\mathit{X}}_n$, $\varphi (·;A)$ is the density function of a p-variate normal distribution with a zero mean vector and covariance matrix A, $\tilde{J}=J^{\left(d\right)}\left({\hat{\mathit{\theta }}}_n^{\left(d\right)}\right)$, ${\tilde{J}}^{-1}=\left({\tilde{J}}^{ij}\right)$, and:

$$\begin{array}{cc}\hfill A_1\left(t_n\right)=& \sum _{i=1}^p\frac{{\partial }_i\pi \left({\hat{\mathit{\theta }}}_n^{\left(d\right)}\right)}{\pi \left({\hat{\mathit{\theta }}}_n^{\left(d\right)}\right)}{t}_i+\frac{1}{6}\sum _{i,j,k}{\overline{D}}_{ijk}\left({\hat{\mathit{\theta }}}_n^{\left(d\right)}\right)t_i{t}_j{t}_k,\hfill \\ \hfill A_2\left(t_n\right)=& \sum _{i,j}\frac{1}{2}\frac{{\partial }_i{\partial }_j\pi \left({\hat{\mathit{\theta }}}_n^{\left(d\right)}\right)}{\pi \left({\hat{\mathit{\theta }}}_n^{\left(d\right)}\right)}(t_i{t}_j-{\tilde{J}}^{ij})-\sum _{i,j,k,\ell }\frac{1}{6}\frac{{\partial }_i\pi \left({\hat{\mathit{\theta }}}_n^{\left(d\right)}\right)}{\pi \left({\hat{\mathit{\theta }}}_n^{\left(d\right)}\right)}{\overline{D}}_{jk\ell }\left({\hat{\mathit{\theta }}}_n^{\left(d\right)}\right)\left(t_i{t}_j{t}_k{t}_{\ell }-3{\tilde{J}}^{ij}{\tilde{J}}^{k\ell }\right)\hfill \\ \hfill & -\sum _{i,j,k,\ell }\frac{1}{24}{\overline{D}}_{ijk\ell }\left({\hat{\mathit{\theta }}}_n^{\left(d\right)}\right)\left(t_i{t}_j{t}_k{t}_{\ell }-3{\tilde{J}}^{ij}{\tilde{J}}^{k\ell }\right)\hfill \\ \hfill & +\sum _{i,j,k,h,g,f}\frac{1}{72}{\overline{D}}_{ijk}{\overline{D}}_{hgf}(2t_i{t}_j{t}_k{t}_h{t}_g{t}_f-15{\tilde{J}}^{ij}{\tilde{J}}^{kh}{\tilde{J}}^{gf}).\hfill \end{array}$$

Proof. 

The proof is given in the Appendix A of [17]. □

As previously mentioned, quasi-posterior distributions depend on the cross-entropy induced by a divergence and a prior distribution. If we have some information about unknown parameters $\mathit{\theta }$, we can use a prior distribution that takes such prior information into account. However, in the absence of prior information, we often use prior distributions known as default or objective priors. Reference [17] proposed the reference prior for quasi-posterior distributions, which is a type of objective prior (see [15]). The reference prior ${\pi }_R$ is obtained by asymptotically maximizing the expected KL divergence between prior and posterior distributions. As a generalization of the reference prior, Reference [19] discussed such priors under a general divergence measure known as the $\alpha$-divergence (see also [20,21]). The reference prior under the $\alpha$-divergence is given by asymptotically maximizing the expected $\alpha$-divergence:

$$H\left(\pi \right)=\mathbb{E}\left[{\mathcal{D}}^{\left(\alpha \right)}({\pi }^{\left(d\right)}\left(\mathit{\theta }\right|{\mathit{X}}_n),\pi \left(\mathit{\theta }\right))\right],$$

where ${\mathcal{D}}^{\left(\alpha \right)}$ is the $\alpha$-divergence defined as:

$$\begin{array}{c}\hfill {\mathcal{D}}^{\left(\alpha \right)}({\pi }^{\left(d\right)}\left(\mathit{\theta }\right|{\mathit{X}}_n),\pi \left(\mathit{\theta }\right))=\frac{1}{\alpha (1-\alpha )}{\int }_{\mathsf{\Theta }}{\left\{1-\left(\frac{\pi \left(\mathit{\theta }\right)}{{\pi }^{\left(d\right)}\left(\mathit{\theta }\right|{\mathit{X}}_n)}\right)\right\}}^{\alpha }{\pi }^{\left(d\right)}\left(\mathit{\theta }\right|{\mathit{X}}_n)\mathrm{d}\mathit{\theta }\end{array}$$

which corresponds to the KL divergence as $\alpha \to 0$, the Hellinger divergence for $\alpha =1/2$, and the ${\chi }^2$-divergence for $\alpha =-1$. Reference [17] derived reference priors with the $\alpha$-divergence under the quasi-posterior based on some kinds of proper scoring rules such as the Tsallis scoring rule and the Hyvärinen scoring rule. We note that the former rule is the same as the density power score of [3] with minor notational modifications.

Theorem 2 

(Giummolè et al. [17]). When $\left|\alpha \right|<1$, the reference prior that asymptotically maximizes the expected α-divergence between the quasi-posterior and prior distributions is given by:

$${\pi }_R\left(\mathit{\theta }\right)\propto det{\left(J^{\left(d\right)}\left(\mathit{\theta }\right)\right)}^{1/2}.$$

The result of Theorem 2 is similar to that of [19,20]. Objective priors such as the above theorem are useful because they can be determined by the data generating model. However, such priors do not have a statistical guarantee when the model is misspecified such as Huber’s $\epsilon$-contamination model. In other words, the reference prior in Theorem 2 depends on data generating distribution g because of $J^{\left(d\right)}\left(\mathit{\theta }\right)=-{\mathbb{E}}_g\left[\partial {\partial }^{\top }{q}^{\left(d\right)}(X_1;\mathit{\theta })\right]$, where $g\left(x\right)=(1-\epsilon )f_{\mathit{\theta }}\left(x\right)+\epsilon \delta \left(x\right)$ when the contamination ratio $\epsilon$ is not small such as for heavy contamination cases. We now consider some objective priors under the $\gamma$-posterior, which is robust against such unknown quantities, in the next section.

3. Main Results

In this section, we show our main results. Our contributions are as follows. We derive moment matching priors for quasi-posterior distributions (Theorem 3). We prove that the proposed priors are robust under the condition on the tail of the contamination distribution (Theorem 4).

3.1. Moment Matching Priors

The moment matching priors proposed by [16] are priors that match the posterior mean and MLE up to the higher order (see also [22]). In this section, we attempt to extend the results of [16] to the context of quasi-posterior distributions. Our goal is to identify a prior such that the difference between the quasi-posterior mean ${\tilde{\mathit{\theta }}}_n^{\left(d\right)}$ and frequentist minimum divergence estimator ${\hat{\mathit{\theta }}}_n^{\left(d\right)}$ converges to zero up to the order of $o\left(n^{-1}\right)$. From Theorem 1, we have the following theorem.

Theorem 3 

Let ${\tilde{\mathit{\theta }}}_n^{\left(d\right)}=({\tilde{\theta }}_1,\dots ,{\tilde{\theta }}_p)$, ${\hat{\mathit{\theta }}}_n^{\left(d\right)}=({\hat{\theta }}_1,\dots ,{\hat{\theta }}_p)$, and $t_n={(t_1,\cdots ,t_p)}^{\top }=\sqrt{n}(\mathit{\theta }-{\hat{\mathit{\theta }}}_n^{\left(d\right)})$. Under the same assumptions as Theorem 1, it holds that:

$$\begin{array}{c}\hfill \begin{array}{c}\hfill n\left({\tilde{\theta }}_{\ell }^{\left(d\right)}-{\hat{\theta }}_{\ell }^{\left(d\right)}\right)\stackrel{p}{\to }\sum _{i=1}^p\frac{{\partial }_i\pi \left({\mathit{\theta }}_g\right)}{\pi \left({\mathit{\theta }}_g\right)}{J}^{i\ell }+\frac{1}{6}\sum _{i,j,k}{g}_{ijk}^{\left(d\right)}\left({\mathit{\theta }}_g\right)\left(J^{ij}{J}^{k\ell }+J^{ik}{J}^{j\ell }+J^{i\ell }{J}^{jk}\right)\end{array}\end{array}$$

as $n\to \infty$, where $J=J^{\left(d\right)}\left({\mathit{\theta }}_g\right)$, $J^{-1}=\left(J^{ij}\right)$, and $g_{ijk}^{\left(d\right)}\left(\mathit{\theta }\right)={\mathbb{E}}_g\left[{\partial }_i{\partial }_j{\partial }_k{q}^{\left(d\right)}(X_1;\mathit{\theta })\right]$. Furthermore, if we set a prior that satisfies:

$$\begin{array}{c}\hfill \frac{{\partial }_{\ell }\pi \left(\mathit{\theta }\right)}{\pi \left(\mathit{\theta }\right)}+\frac{1}{2}\sum _{i,j}{g}_{ij\ell }^{\left(d\right)}\left(\mathit{\theta }\right)J^{ij}\left(\mathit{\theta }\right)=0\end{array}$$

(4)

for all $\ell =1,\dots ,p$, then it holds that:

$$n\left({\tilde{\theta }}_{\ell }^{\left(d\right)}-{\hat{\theta }}_{\ell }^{\left(d\right)}\right)\stackrel{p}{\to }0$$

for $\ell =1,\dots ,p$ as $n\to \infty$, where ${\left\{J^{\left(d\right)}\left(\mathit{\theta }\right)\right\}}^{-1}=\left(J^{ij}\left(\mathit{\theta }\right)\right)$.

Hereafter, the prior that satisfies Equation (4) up to the order of $o_p\left(n^{-1}\right)$ for all $\ell =1,\dots ,p$ is referred to as a moment matching prior, and we denote it by ${\pi }_M$.

Proof. 

From the asymptotic expansion of the posterior density (3), we have the asymptotic expansion of the posterior mean for ${\theta }_{\ell }$ as:

$$\begin{array}{c}\hfill \begin{array}{cc}\hfill {\tilde{\theta }}_{\ell }^{\left(d\right)}& ={\int }_{\mathsf{\Theta }}{\theta }_{\ell }{\pi }^{\left(d\right)}\left(\mathit{\theta }\right|{\mathit{X}}_n)\mathrm{d}\mathit{\theta }\hfill \\ \hfill & ={\hat{\theta }}_{\ell }^{\left(d\right)}+\frac{1}{\sqrt{n}}{\int }_{{\mathbb{R}}^p}{t}_{\ell }{\pi }^{*\left(d\right)}\left(t_n\right|{\mathit{X}}_n)\mathrm{d}{t}_n\hfill \\ \hfill & ={\hat{\theta }}_{\ell }^{\left(d\right)}+\frac{1}{n}{\int }_{{\mathbb{R}}^p}{t}_{\ell }\varphi \left(t_n;{\tilde{J}}^{-1}\right)A_1\left(t_n\right)\mathrm{d}{t}_n+O_p\left(n^{-3/2}\right)\hfill \end{array}\end{array}$$

(5)

for $\ell =1,\dots ,p$. The integral in the above equation is calculated by:

$$\begin{array}{c}\hfill \begin{array}{cc}\hfill {\int }_{{\mathbb{R}}^p}{t}_{\ell }{A}_1\left(t_n\right)\varphi \left(t_n;{\tilde{J}}^{-1}\right)\mathrm{d}{t}_n=& \sum _{i=1}^p\frac{{\partial }_i\pi \left({\hat{\mathit{\theta }}}_n^{\left(d\right)}\right)}{\pi \left({\hat{\mathit{\theta }}}_n^{\left(d\right)}\right)}{\int }_{{\mathbb{R}}^p}{t}_i{t}_{\ell }\varphi \left(t_n;{\tilde{J}}^{-1}\right)\mathrm{d}{t}_n\hfill \\ \hfill & +\frac{1}{6}\sum _{i,j,k}{\overline{D}}_{ijk}\left({\hat{\mathit{\theta }}}_n^{\left(d\right)}\right){\int }_{{\mathbb{R}}^p}{t}_i{t}_j{t}_k{t}_{\ell }\varphi \left(t_n;{\tilde{J}}^{-1}\right)\mathrm{d}{t}_n\hfill \\ \hfill =& \sum _{i=1}^p\frac{{\partial }_i\pi \left({\hat{\mathit{\theta }}}_n^{\left(d\right)}\right)}{\pi \left({\hat{\mathit{\theta }}}_n^{\left(d\right)}\right)}{\tilde{J}}^{i\ell }\hfill \\ \hfill & +\frac{1}{6}\sum _{i,j,k}{\overline{D}}_{ijk}\left({\hat{\mathit{\theta }}}_n^{\left(d\right)}\right)\left({\tilde{J}}^{ij}{\tilde{J}}^{k\ell }+{\tilde{J}}^{ik}{\tilde{J}}^{j\ell }+{\tilde{J}}^{i\ell }{\tilde{J}}^{jk}\right)+o_p\left(1\right).\hfill \end{array}\end{array}$$

(6)

From (5) and (6), we have:

$${\tilde{\theta }}_{\ell }^{\left(d\right)}-{\hat{\theta }}_{\ell }^{\left(d\right)}=\sum _{i=1}^p\frac{{\partial }_i\pi \left({\hat{\mathit{\theta }}}_n^{\left(d\right)}\right)}{\pi \left({\hat{\mathit{\theta }}}_n^{\left(d\right)}\right)}{\tilde{J}}^{i\ell }+\frac{1}{6n}\sum _{i,j,k}{\overline{D}}_{ijk}\left({\hat{\mathit{\theta }}}_n^{\left(d\right)}\right)\left({\tilde{J}}^{ij}{\tilde{J}}^{k\ell }+{\tilde{J}}^{ik}{\tilde{J}}^{j\ell }+{\tilde{J}}^{i\ell }{\tilde{J}}^{jk}\right)+O_p\left(n^{-3/2}\right)$$

for $\ell =1,\dots ,p$. By using the consistency of the estimator ${\hat{\mathit{\theta }}}_n^{\left(d\right)}$, we then have the following asymptotic difference between ${\hat{\theta }}_{\ell }^{\left(d\right)}$ and ${\hat{\theta }}_{\ell }^{\left(d\right)}$:

$$\begin{array}{c}\hfill n\left({\tilde{\theta }}_{\ell }^{\left(d\right)}-{\hat{\theta }}_{\ell }^{\left(d\right)}\right)\stackrel{p}{\to }\sum _{i=1}^p\frac{{\partial }_i\pi \left({\mathit{\theta }}_g\right)}{\pi \left({\mathit{\theta }}_g\right)}{J}^{i\ell }+\frac{1}{6}\sum _{i,j,k}{g}_{ijk}^{\left(d\right)}\left({\mathit{\theta }}_g\right)\left(J^{ij}{J}^{k\ell }+J^{ik}{J}^{j\ell }+J^{i\ell }{J}^{jk}\right)\end{array}$$

as $n\to \infty$ for $\ell =1,\dots ,p$. □

In general, it is not easy to obtain the moment matching priors explicitly. Two examples are given as follows.

Example 1.

When $p=1$, the moment matching prior is given by:

$${\pi }_M\left(\theta \right)=Cexp\left\{-{\int }^{\theta }\frac{g_3^{\left(d\right)}\left(t\right)}{2J^{\left(d\right)}\left(t\right)}\mathrm{d}t\right\}$$

for a constant C, where $g_3$ is a third derivation of g. This prior is very similar to that of [16], but the quantities $g_3^{\left(d\right)}\left(t\right)$ and $J^{\left(d\right)}\left(t\right)$ are different from it.

Example 2.

When $p=2$, we put:

$$u_{\ell }({\theta }_1,{\theta }_2)=\sum _{i,j}{g}_{ij\ell }^{\left(d\right)}\left(\mathit{\theta }\right)J^{ij}\left(\mathit{\theta }\right)(\ell =1,2),$$

where $\mathit{\theta }={({\theta }_1,{\theta }_2)}^{\top }$. If $u_{\ell }({\theta }_1,{\theta }_2)$ only depends on ${\theta }_{\ell }$ for all $\ell =1,2$ and does not depend on other parameters ${\theta }_k(k\ne \ell )$, we have:

$$\begin{array}{c}\hfill u_1({\theta }_1,{\theta }_2)\equiv u_1\left({\theta }_1\right),u_2({\theta }_1,{\theta }_2)\equiv u_2\left({\theta }_2\right).\end{array}$$

Then, we can solve the differential equation given by (4), and the moment matching prior is obtained by

$${\pi }_M({\theta }_1,{\theta }_2)\propto exp\left\{-\frac{1}{2}{\int }^{{\theta }_1}{u}_1\left(t_1\right)\mathrm{d}{t}_1\right\}exp\left\{-\frac{1}{2}{\int }^{{\theta }_2}{u}_2\left(t_2\right)\mathrm{d}{t}_2\right\}.$$

3.2. Robustness of Objective Priors

For data that may be heavily contaminated, we cannot assume that the contamination ratio $\epsilon$ is approximately zero. In general, reference and moment matching priors depend on the contamination ratio and distribution. Therefore, we cannot directly use such objective priors for the quasi-posterior distributions because the contamination ratio $\epsilon$ and the contamination distribution $\delta \left(x\right)$ are unknown. In this subsection, we prove that priors based on the $\gamma$-divergence are robust against these unknown quantities. In addition to (1), we assume the following condition of the contamination distribution:

$$\begin{array}{c}\hfill {\nu }_{\mathit{\theta }}={\left\{{\int }_{\mathsf{\Omega }}\delta \left(x\right)f_{\mathit{\theta }}{\left(x\right)}^{{\gamma }_0}\mathrm{d}x\right\}}^{1/{\gamma }_0}\approx 0\end{array}$$

(7)

for all $\mathit{\theta }\in \mathsf{\Theta }$ and an appropriately large constant ${\gamma }_0>0$ (see also [5]). Note that the assumption (7) is also a basis to prove the robustness against outliers for the minimum $\gamma$-divergence estimator in [5]. Then, we have the following theorem.

Theorem 4 

Assume the condition (7). Let:

$$\begin{array}{c}\hfill q^{\left(\gamma \right)}(x;\mathit{\theta }):=q^{\left({\tilde{d}}_{\gamma }\right)}(x;\mathit{\theta })=\frac{1}{\gamma }{f}_{\mathit{\theta }}{\left(x\right)}^{\gamma }{\left\{{\int }_{\mathsf{\Omega }}{f}_{\mathit{\theta }}{\left(y\right)}^{1+\gamma }\mathrm{d}y\right\}}^{-\gamma /(1+\gamma )},\end{array}$$

and let:

$$\begin{array}{cc}\hfill h_{ij}^{\left(\gamma \right)}\left(\mathit{\theta }\right)& =-{\mathbb{E}}_{f_{\mathit{\theta }}}\left[{\partial }_i{\partial }_j{q}^{\left(\gamma \right)}(X_1;\mathit{\theta })\right],\hfill \\ \hfill {\tilde{g}}_{ijk}^{\left(\gamma \right)}\left(\mathit{\theta }\right)& ={\mathbb{E}}_{f_{\mathit{\theta }}}\left[{\partial }_i{\partial }_j{\partial }_k{q}^{\left(\gamma \right)}(X_1;\mathit{\theta })\right].\hfill \end{array}$$

Then, it holds that:

$$\begin{array}{c}\hfill J_{ij}^{\left(\gamma \right)}\left(\mathit{\theta }\right)=-{\mathbb{E}}_g\left[{\partial }_i{\partial }_j{q}^{\left(\gamma \right)}(X_1;\mathit{\theta })\right]=(1-\epsilon )h_{ij}^{\left(\gamma \right)}\left(\mathit{\theta }\right)+O\left(\epsilon {\nu }^{\gamma }\right),\\ \hfill g_{ijk}^{\left(\gamma \right)}\left(\mathit{\theta }\right)={\mathbb{E}}_g\left[{\partial }_i{\partial }_j{\partial }_k{q}^{\left(\gamma \right)}(X_1;\mathit{\theta })\right]=(1-\epsilon ){\tilde{g}}_{ijk}^{\left(\gamma \right)}\left(\mathit{\theta }\right)+O\left(\epsilon {\nu }^{\gamma }\right),\end{array}$$

(8)

for $\gamma +1\le {\gamma }_0$, where $\nu :=max\{{\nu }_f,{sup}_{\mathit{\theta }\in \mathsf{\Theta }}{\nu }_{\mathit{\theta }}\}$. The notation $O\left(\epsilon {\nu }^{\gamma }\right)$ is the same use as that of [5]. Furthermore, from the above results, the reference prior and Equation (4) are approximately given by:

$$\begin{array}{c}\hfill {\pi }_R\left(\mathit{\theta }\right)\propto det{\left(H^{\left(\gamma \right)}\left(\mathit{\theta }\right)\right)}^{1/2},\\ \hfill \frac{{\partial }_{\ell }\pi \left(\mathit{\theta }\right)}{\pi \left(\mathit{\theta }\right)}+\frac{1}{2}\sum _{i,j}{\tilde{g}}_{ij\ell }^{\left(\gamma \right)}\left(\mathit{\theta }\right)h^{ij}\left(\mathit{\theta }\right)=0,\end{array}$$

(9)

where $H^{\left(\gamma \right)}\left(\mathit{\theta }\right)=\left(h_{ij}^{\left(\gamma \right)}\left(\theta \right)\right)$ and ${\left\{H^{\left(\gamma \right)}\left(\mathit{\theta }\right)\right\}}^{-1}=\left(h^{ij}\left(\mathit{\theta }\right)\right)$.

Proof. 

Put $\ell \left(x\right)=log{f}_{\mathit{\theta }}\left(x\right)$, ${\ell }_i\left(x\right)={\partial }_{i}log{f}_{\mathit{\theta }}\left(x\right)$, ${\ell }_{ij}\left(x\right)={\partial }_i{\partial }_{j}log{f}_{\mathit{\theta }}\left(x\right)$ and ${\ell }_{ijk}\left(x\right)={\partial }_i{\partial }_j{\partial }_{k}log{f}_{\mathit{\theta }}\left(x\right)$. First, from Hölder’s inequality and Lyapunonv’s inequality, it holds that:

$$\begin{array}{c}{\int }_{\mathsf{\Omega }}\left|\delta \left(x\right)f_{\mathit{\theta }}{\left(x\right)}^{\gamma }{\ell }_i\left(x\right)\right|\mathrm{d}x\le {\nu }^{\gamma }{\left({\int }_{\mathsf{\Omega }}{\left|{\ell }_i\left(x\right)\right|}^{1+\gamma }\delta \left(x\right)\mathrm{d}x\right)}^{1/(1+\gamma )},\hfill \\ {\int }_{\mathsf{\Omega }}\left|\delta \left(x\right)f_{\mathit{\theta }}{\left(x\right)}^{\gamma }{\ell }_i\left(x\right){\ell }_j\left(x\right)\right|\mathrm{d}x\le {\nu }^{\gamma }{\left({\int }_{\mathsf{\Omega }}{\left|{\ell }_i\left(x\right){\ell }_j\left(x\right)\right|}^{1+\gamma }\delta \left(x\right)\mathrm{d}x\right)}^{1/(1+\gamma )},\hfill \\ {\int }_{\mathsf{\Omega }}\left|\delta \left(x\right)f_{\mathit{\theta }}{\left(x\right)}^{\gamma }{\ell }_{ij}\left(x\right)\right|\mathrm{d}x\le {\nu }^{\gamma }{\left({\int }_{\mathsf{\Omega }}{\left|{\ell }_{ij}\left(x\right)\right|}^{1+\gamma }\delta \left(x\right)\mathrm{d}x\right)}^{1/(1+\gamma )},\hfill \\ {\int }_{\mathsf{\Omega }}\left|\delta \left(x\right)f_{\mathit{\theta }}{\left(x\right)}^{\gamma }{\ell }_{ijk}\left(x\right)\right|\mathrm{d}x\le {\nu }^{\gamma }{\left({\int }_{\mathsf{\Omega }}{\left|{\ell }_{ijk}\left(x\right)\right|}^{1+\gamma }\delta \left(x\right)\mathrm{d}x\right)}^{1/(1+\gamma )},\hfill \\ {\int }_{\mathsf{\Omega }}\left|\delta \left(x\right)f_{\mathit{\theta }}{\left(x\right)}^{\gamma }{\ell }_{ij}\left(x\right){\ell }_k\left(x\right)\right|\mathrm{d}x\le {\nu }^{\gamma }{\left({\int }_{\mathsf{\Omega }}{\left|{\ell }_{ij}\left(x\right){\ell }_k\left(x\right)\right|}^{1+\gamma }\delta \left(x\right)\mathrm{d}x\right)}^{1/(1+\gamma )},\hfill \\ {\int }_{\mathsf{\Omega }}\left|\delta \left(x\right)f_{\mathit{\theta }}{\left(x\right)}^{\gamma }{\ell }_i\left(X\right){\ell }_j\left(X_1\right){\ell }_k\left(X_1\right)\right|\mathrm{d}x\le {\nu }^{\gamma }{\left({\int }_{\mathsf{\Omega }}{\left|{\ell }_i\left(x\right){\ell }_j\left(x\right){\ell }_k\left(x\right)\right|}^{1+\gamma }\delta \left(x\right)\mathrm{d}x\right)}^{1/(1+\gamma )}\hfill \end{array}$$

(10)

for $i,j,k=1,\dots ,p$. Using (10) and the results in Appendix A, we have:

$$\begin{array}{cc}\hfill \left|{\int }_{\mathsf{\Omega }}\delta \left(x\right){\partial }_i{\partial }_j{q}^{\left(\gamma \right)}(x;\mathit{\theta })\mathrm{d}x\right|\le & \parallel f_{\mathit{\theta }}{\parallel }_{\gamma +1}^{-\gamma }\gamma {\int }_{\mathsf{\Omega }}\left|\delta \left(x\right)f_{\mathit{\theta }}{\left(x\right)}^{\gamma }{\ell }_i\left(x\right){\ell }_j\left(x\right)\right|\mathrm{d}x\hfill \\ \hfill & +\parallel f_{\mathit{\theta }}{\parallel }_{\gamma +1}^{-\gamma }{\int }_{\mathsf{\Omega }}\left|\delta \left(x\right)f_{\mathit{\theta }}{\left(x\right)}^{\gamma }{\ell }_{ij}\left(x\right)\right|\mathrm{d}x\hfill \\ \hfill & +\gamma S_i{\parallel f_{\mathit{\theta }}\parallel }_{\gamma +1}^{-1-2\gamma }{\int }_{\mathsf{\Omega }}\left|\delta \left(x\right)f_{\mathit{\theta }}{\left(x\right)}^{\gamma }{\ell }_j\left(x\right)\right|\mathrm{d}x\hfill \\ \hfill & +\gamma S_j{\parallel f_{\mathit{\theta }}\parallel }_{\gamma +1}^{-1-2\gamma }{\int }_{\mathsf{\Omega }}\left|\delta \left(x\right)f_{\mathit{\theta }}{\left(x\right)}^{\gamma }{\ell }_i\left(x\right)\right|\mathrm{d}x\hfill \\ \hfill & +\frac{(1+2\gamma )}{\parallel f_{\mathit{\theta }}{\parallel }_{\gamma +1}^{2+3\gamma }}{S}_i{S}_j{\int }_{\mathsf{\Omega }}\left|\delta \left(x\right)f_{\mathit{\theta }}{\left(x\right)}^{\gamma }\right|\mathrm{d}x\hfill \\ \hfill & +\parallel f_{\mathit{\theta }}{\parallel }_{\gamma +1}^{-1-2\gamma }{\int }_{\mathsf{\Omega }}\left|\delta \left(x\right)f_{\mathit{\theta }}{\left(x\right)}^{\gamma }\right|\mathrm{d}x{\int }_{\mathsf{\Omega }}{f}_{\mathit{\theta }}{\left(y\right)}^{\gamma +1}{s}_{ij}\left(y\right)\mathrm{d}y,\hfill \\ \hfill =& O\left({\nu }^{\gamma }\right),\hfill \end{array}$$

where:

$$\begin{array}{cc}\hfill s_{ij}\left(y\right)=& (\gamma +1){\ell }_i\left(y\right){\ell }_j\left(y\right)+{\ell }_{ij}\left(y\right),\hfill \\ \hfill S_i=& {\int }_{\mathsf{\Omega }}{f}_{\mathit{\theta }}{\left(y\right)}^{\gamma +1}{\ell }_i\left(y\right)\mathrm{d}y\hfill \end{array}$$

for $i,j=1,\dots ,p$. Similarly, it also holds that:

$$\begin{array}{cc}\hfill & \int \delta \left(x\right){\partial }_i{\partial }_j{\partial }_k{q}^{\left(\gamma \right)}(x;\mathit{\theta })\mathrm{d}x=O\left({\nu }^{\gamma }\right)\hfill \end{array}$$

for $i,j,k=1,\dots ,p$. Since,

$$\begin{array}{cc}\hfill J_{ij}^{\left(\gamma \right)}\left(\mathit{\theta }\right)=& -{\mathbb{E}}_g\left[{\partial }_i{\partial }_j{q}^{\left(\gamma \right)}(X_1;\mathit{\theta })\right]=-(1-\epsilon )h_{ij}^{\left(\gamma \right)}\left(\mathit{\theta }\right)-\epsilon {\int }_{\mathsf{\Omega }}\delta \left(x\right){\partial }_i{\partial }_j{q}^{\left(\gamma \right)}(x;\mathit{\theta })\mathrm{d}x,\hfill \\ \hfill g_{ijk}^{\left(\gamma \right)}\left(\mathit{\theta }\right)=& {\mathbb{E}}_g\left[{\partial }_i{\partial }_j{\partial }_k{q}^{\left(\gamma \right)}(X_1;\mathit{\theta })\right]=(1-\epsilon ){\tilde{g}}_{ijk}^{\left(\gamma \right)}\left(\mathit{\theta }\right)+\epsilon \int \delta \left(x\right){\partial }_i{\partial }_j{\partial }_k{q}^{\left(\gamma \right)}(x;\mathit{\theta })\mathrm{d}x,\hfill \end{array}$$

the proof of (8) is complete. It is also easy to see the result of (9) from (8). □

It should be noted that (8) looks like the results for Theorem 5.1 in [5]. However, $q^{\left(\gamma \right)}(x;\mathit{\theta })$, and its derivative functions are different formulae from those of [5], so that the derivative functions and the proof of (8) are given in the Appendix A. Theorem 4 shows that expectations on the right-hand side of $J_{ij}^{\left(\gamma \right)}\left(\mathit{\theta }\right)$ and $g_{ijk}^{\left(\gamma \right)}\left(\mathit{\theta }\right)$ only depend on the underlying model $f_{\mathit{\theta }}$, but do not depend on the contamination distribution. Furthermore, reference and moment matching priors for the $\gamma$-posterior are obtained by the parametric model $f_{\mathit{\theta }}$, that is, these do not depend on the contamination ratio and the contamination distribution. For example, for a normal distribution $N(\mu ,{\sigma }^2)$, reference and moment matching priors are given by:

$$\begin{array}{c}{\pi }_R^{\left(\gamma \right)}(\mu ,\sigma )={\sigma }^{-3+1/(1+\gamma )}+O\left(\epsilon {\nu }^{\gamma }\right),\hfill \\ {\pi }_M^{\left(\gamma \right)}(\mu ,\sigma )={\sigma }^{-(\gamma +7)/\left\{2\right(1+\gamma \left)\right\}}+O\left(\epsilon {\nu }^{\gamma }\right).\hfill \end{array}$$

(11)

However, reference and moment matching priors under the $R^{\left(\alpha \right)}$-posterior depend on unknown quantities in the data generating distribution unless $\epsilon \approx 0$, since $J_{ij}^{\left(\alpha \right)}\left(\mathit{\theta }\right)$ and $g_{ijk}^{\left(\alpha \right)}\left(\mathit{\theta }\right)$ have the following forms:

$$\begin{array}{cc}\hfill J_{ij}^{\left(\alpha \right)}\left(\mathit{\theta }\right)=& -{\mathbb{E}}_g\left[{\partial }_i{\partial }_j{q}^{\left(\alpha \right)}(X_1;\mathit{\theta })\right]\hfill \\ \hfill =& -(1-\epsilon ){\mathbb{E}}_{f_{\mathit{\theta }}}\left[{\partial }_i{\partial }_j{q}^{\left(\alpha \right)}(X_1;\mathit{\theta })\right]-\frac{\epsilon }{1+\alpha }{\int }_{\mathsf{\Omega }}{\partial }_i{\partial }_j{f}_{\mathit{\theta }}{\left(x\right)}^{1+\alpha }\mathrm{d}x+O\left(\epsilon {\nu }^{\alpha }\right),\hfill \\ \hfill g_{ijk}^{\left(\alpha \right)}\left(\mathit{\theta }\right)=& {\mathbb{E}}_g\left[{\partial }_i{\partial }_j{\partial }_k{q}^{\left(\alpha \right)}(X_1;\mathit{\theta })\right]\hfill \\ \hfill =& (1-\epsilon ){\mathbb{E}}_{f_{\mathit{\theta }}}\left[{\partial }_i{\partial }_j{\partial }_k{q}^{\left(\alpha \right)}(X_1;\mathit{\theta })\right]+\frac{\epsilon }{1+\alpha }{\int }_{\mathsf{\Omega }}{\partial }_i{\partial }_j{\partial }_k{f}_{\mathit{\theta }}{\left(x\right)}^{1+\alpha }\mathrm{d}x+O\left(\epsilon {\nu }^{\alpha }\right),\hfill \end{array}$$

where:

$$\begin{array}{c}\hfill q^{\left(\alpha \right)}(x;\mathit{\theta }):=q^{\left(d_{\alpha }\right)}(x;\mathit{\theta })=\frac{1}{\alpha }{f}_{\mathit{\theta }}{\left(x\right)}^{\alpha }-\frac{1}{1+\alpha }{\int }_{\mathsf{\Omega }}{f}_{\mathit{\theta }}{\left(y\right)}^{1+\alpha }\mathrm{d}y.\end{array}$$

The priors given by (11) can be practically used under the condition (7) even if the contamination ratio $\epsilon$ is not small.

4. Simulation Studies

4.1. Setting and Results

We present the performance of posterior means under reference and moment matching priors through some simulation studies. In this section, we assume that the parametric model is the normal distribution with mean $\mu$ and variance ${\sigma }^2$ and consider the joint estimation problem for $\mu$ and ${\sigma }^2$. We assume that the true values of $\mu$ and ${\sigma }^2$ are zero and one, respectively. We also assume that the contamination distribution is the normal distribution with mean $\nu$ and variance one. In other words, the data generating distribution is expressed by:

$$g\left(x\right)=(1-\epsilon )N(0,1)+\epsilon N(\nu ,1),$$

where $\epsilon$ is the contamination ratio and n is the sample size. We compare the performances of estimators in terms of empirical bias and mean squared error (MSE) among three methods, which include the ordinary KL divergence-based posterior, $R^{\left(\alpha \right)}$-posterior, and $\gamma$-posterior (our proposal). We also employ three prior distributions for $(\mu ,\sigma )$, namely (i) uniform prior, (ii) reference prior, and (iii) moment matching prior.

Since exact calculations of posterior means are not easy, we use the importance sampling Monte Carlo algorithm using the proposal distributions $N(\overline{x},s^2)$ for $\mu$ and $\mathrm{IG}(6,5s)$ for $\sigma$ (the inverse gamma distribution with parameters a and b is denoted by $\mathrm{IG}(a,b)$), where $\overline{x}=n^{-1}{\sum }_{i=1}^n{x}_i$ and $s^2={(n-1)}^{-1}{\sum }_{i=1}^n{(x_i-\overline{x})}^2$ (for the details of the importance sampling, see, e.g., [23]). We carry out the importance sampling with 10,000 steps, and we compute the empirical bias and MSE for posterior means $(\hat{\mu },\hat{\sigma })$ of $(\mu ,\sigma )$ by 10,000 iterations. The simulation results are reported in Table 1, Table 2, Table 3 and Table 4. The reference and the moment matching priors for the $\gamma$-posterior are given by (11), and those for the $R^{\left(\alpha \right)}$-posterior are “formally” given as follows:

$$\begin{array}{c}\hfill {\pi }_M^{\left(\alpha \right)}(\mu ,\sigma )\propto {\sigma }^{-2-\alpha },{\pi }_M^{\left(\alpha \right)}(\mu ,\sigma )\propto {\sigma }^{C_M/2},\end{array}$$

(12)

where $C_M$ is a constant given by:

$$\begin{array}{c}\hfill C_M=-\frac{2+{\alpha }^2}{(1+\alpha )}+\frac{\alpha {(1+\alpha )}^3(2+\alpha )+(10-{\alpha }^2(-2+\alpha (5+\alpha (3+\alpha )))){\pi }^{\alpha /2}}{(1+\alpha )(-\alpha {(1+\alpha )}^2+(-2+\alpha +{\alpha }^2+{\alpha }^3){\pi }^{\alpha /2})}.\end{array}$$

The term “formally” means that since the reference and the moment matching priors for the $R^{\left(\alpha \right)}$-posterior strictly depend on an unknown contamination ratio and contamination distribution, we set $\epsilon =0$ in these priors. On the other hand, our proposed objective priors do not need such an assumption, but we assume only the condition (7). We note that [17] also used the same formal reference prior in their simulation studies.

Table 1. Empirical biases of the posterior means for $\mu$.

$\mathit{\epsilon }$	$\mathit{n}$	Bayes	${\mathit{R}}^{\left(\mathit{\alpha }\right)}$-Posterior				$\mathit{\gamma }$-Posterior
		$\mathit{\alpha },\mathit{\gamma }\to \mathbf{0.0}$	$\mathit{\alpha }=\mathbf{0.2}$	$\mathit{\alpha }=\mathbf{0.3}$	$\mathit{\alpha }=\mathbf{0.5}$	$\mathit{\alpha }=\mathbf{0.7}$	$\mathit{\gamma }=\mathbf{0.2}$	$\mathit{\gamma }=\mathbf{0.3}$	$\mathit{\gamma }=\mathbf{0.5}$	$\mathit{\gamma }=\mathbf{0.7}$
Uniform prior
$0.00$	20	$-0.002$	$-0.003$	$-0.003$	$-0.002$	$0.001$	$-0.003$	$-0.003$	$-0.003$	$-0.002$
$0.00$	50	$-0.002$	$-0.001$	$-0.001$	$-0.001$	$0.000$	$-0.001$	$-0.001$	$-0.001$	$0.000$
$0.00$	100	$0.000$	$0.000$	$0.000$	$0.000$	$0.001$	$0.000$	$0.000$	$0.000$	$0.001$
$0.05$	20	$0.298$	$0.109$	$0.075$	$0.098$	$0.172$	$0.104$	$0.064$	$0.046$	$0.060$
$0.05$	50	$0.301$	$0.053$	$0.020$	$0.009$	$0.016$	$0.051$	$0.017$	$0.004$	$0.002$
$0.05$	100	$0.301$	$0.038$	$0.012$	$0.004$	$0.002$	$0.036$	$0.011$	$0.003$	$0.001$
$0.20$	20	$1.192$	$0.917$	$0.800$	$0.815$	$0.973$	$0.908$	$0.755$	$0.596$	$0.615$
$0.20$	50	$1.198$	$0.869$	$0.638$	$0.362$	$0.478$	$0.864$	$0.600$	$0.215$	$0.112$
$0.20$	100	$1.201$	$0.862$	$0.578$	$0.158$	$0.108$	$0.859$	$0.537$	$0.065$	$0.015$
Reference prior
$0.00$	20	$-0.002$	$-0.003$	$-0.004$	$-0.004$	$-0.003$	$-0.003$	$-0.004$	$-0.004$	$-0.004$
$0.00$	50	$-0.002$	$-0.001$	$-0.001$	$-0.001$	$0.000$	$-0.001$	$-0.001$	$-0.001$	$0.000$
$0.00$	100	$0.000$	$0.000$	$0.000$	$0.000$	$0.001$	$0.000$	$0.000$	$0.000$	$0.001$
$0.05$	20	$0.298$	$0.072$	$0.033$	$0.016$	$0.018$	$0.070$	$0.030$	$0.010$	$0.006$
$0.05$	50	$0.301$	$0.041$	$0.013$	$0.002$	$0.001$	$0.040$	$0.011$	$0.001$	$-0.001$
$0.05$	100	$0.301$	$0.033$	$0.010$	$0.003$	$0.001$	$0.032$	$0.009$	$0.002$	$0.001$
$0.20$	20	$1.192$	$0.808$	$0.558$	$0.295$	$0.293$	$0.803$	$0.537$	$0.227$	$0.152$
$0.20$	50	$1.198$	$0.820$	$0.504$	$0.143$	$0.079$	$0.817$	$0.473$	$0.085$	$0.023$
$0.20$	100	$1.201$	$0.838$	$0.495$	$0.071$	$0.027$	$0.836$	$0.457$	$0.029$	$0.006$
Moment matching prior
$0.00$	20	$-0.002$	$-0.003$	$-0.004$	$-0.004$	$-0.003$	$-0.003$	$-0.004$	$-0.004$	$-0.004$
$0.00$	50	$-0.002$	$-0.001$	$-0.001$	$-0.001$	$0.000$	$-0.001$	$-0.001$	$-0.001$	$0.000$
$0.00$	100	$0.000$	$0.000$	$0.000$	$0.001$	$0.001$	$0.000$	$0.000$	$0.000$	$0.001$
$0.05$	20	$0.298$	$0.059$	$0.025$	$0.010$	$0.008$	$0.059$	$0.024$	$0.009$	$0.007$
$0.05$	50	$0.301$	$0.037$	$0.011$	$0.002$	$-0.001$	$0.036$	$0.010$	$0.001$	$-0.001$
$0.05$	100	$0.301$	$0.031$	$0.009$	$0.002$	$0.001$	$0.030$	$0.009$	$0.002$	$0.001$
$0.20$	20	$1.192$	$0.759$	$0.486$	$0.220$	$0.196$	$0.759$	$0.481$	$0.210$	$0.165$
$0.20$	50	$1.198$	$0.799$	$0.462$	$0.111$	$0.043$	$0.797$	$0.441$	$0.079$	$0.025$
$0.20$	100	$1.201$	$0.828$	$0.468$	$0.058$	$0.018$	$0.827$	$0.435$	$0.028$	$0.006$

Table 2. Empirical biases of the posterior means for $\sigma$.

$\mathit{\epsilon }$	$\mathit{n}$	Bayes	${\mathit{R}}^{\left(\mathit{\alpha }\right)}$-Posterior				$\mathit{\gamma }$-Posterior
		$\mathit{\alpha },\mathit{\gamma }\to \mathbf{0.0}$	$\mathit{\alpha }=\mathbf{0.2}$	$\mathit{\alpha }=\mathbf{0.3}$	$\mathit{\alpha }=\mathbf{0.5}$	$\mathit{\alpha }=\mathbf{0.7}$	$\mathit{\gamma }=\mathbf{0.2}$	$\mathit{\gamma }=\mathbf{0.3}$	$\mathit{\gamma }=\mathbf{0.5}$	$\mathit{\gamma }=\mathbf{0.7}$
Uniform prior
$0.00$	20	$0.058$	$0.148$	$0.225$	$0.733$	$2.089$	$0.136$	$0.184$	$0.330$	$0.620$
$0.00$	50	$0.022$	$0.049$	$0.067$	$0.122$	$0.263$	$0.046$	$0.058$	$0.085$	$0.116$
$0.00$	100	$0.011$	$0.024$	$0.031$	$0.053$	$0.088$	$0.022$	$0.028$	$0.039$	$0.051$
$0.05$	20	$0.669$	$0.438$	$0.476$	$1.620$	$4.335$	$0.404$	$0.370$	$0.540$	$1.109$
$0.05$	50	$0.660$	$0.203$	$0.144$	$0.188$	$0.475$	$0.189$	$0.116$	$0.110$	$0.139$
$0.05$	100	$0.652$	$0.134$	$0.078$	$0.087$	$0.135$	$0.123$	$0.061$	$0.049$	$0.058$
$0.20$	20	$1.732$	$1.848$	$2.086$	$5.500$	$9.627$	$1.769$	$1.727$	$2.207$	$3.833$
$0.20$	50	$1.653$	$1.558$	$1.304$	$1.098$	$3.158$	$1.533$	$1.182$	$0.573$	$0.454$
$0.20$	100	$1.626$	$1.508$	$1.151$	$0.506$	$0.563$	$1.495$	$1.042$	$0.198$	$0.113$
Reference prior
$0.00$	20	$-0.001$	$0.009$	$0.006$	$-0.007$	$-0.013$	$0.007$	$-0.001$	$-0.041$	$-0.117$
$0.00$	50	$0.000$	$0.003$	$0.002$	$-0.004$	$-0.010$	$0.003$	$0.000$	$-0.012$	$-0.036$
$0.00$	100	$0.000$	$0.002$	$0.001$	$-0.002$	$-0.006$	$0.002$	$0.000$	$-0.005$	$-0.016$
$0.05$	20	$0.576$	$0.173$	$0.093$	$0.066$	$0.097$	$0.161$	$0.069$	$0.000$	$-0.051$
$0.05$	50	$0.625$	$0.119$	$0.050$	$0.028$	$0.029$	$0.110$	$0.035$	$-0.003$	$-0.030$
$0.05$	100	$0.635$	$0.096$	$0.039$	$0.024$	$0.026$	$0.088$	$0.026$	$0.000$	$-0.014$
$0.20$	20	$1.580$	$1.281$	$0.954$	$0.659$	$0.697$	$1.258$	$0.877$	$0.427$	$0.303$
$0.20$	50	$1.598$	$1.367$	$0.917$	$0.375$	$0.324$	$1.354$	$0.832$	$0.181$	$0.071$
$0.20$	100	$1.599$	$1.421$	$0.937$	$0.241$	$0.196$	$1.413$	$0.839$	$0.068$	$0.014$
Moment matching prior
$0.00$	20	$-0.039$	$-0.036$	$-0.044$	$-0.083$	$-0.186$	$-0.034$	$-0.039$	$-0.061$	$-0.090$
$0.00$	50	$-0.015$	$-0.014$	$-0.016$	$-0.029$	$-0.067$	$-0.013$	$-0.014$	$-0.019$	$-0.027$
$0.00$	100	$-0.007$	$-0.006$	$-0.007$	$-0.014$	$-0.032$	$-0.006$	$-0.006$	$-0.008$	$-0.012$
$0.05$	20	$0.516$	$0.093$	$0.021$	$-0.029$	$-0.113$	$0.089$	$0.016$	$-0.021$	$-0.023$
$0.05$	50	$0.601$	$0.089$	$0.026$	$-0.002$	$-0.037$	$0.083$	$0.017$	$-0.011$	$-0.021$
$0.05$	100	$0.623$	$0.082$	$0.027$	$0.010$	$-0.005$	$0.075$	$0.017$	$-0.003$	$-0.010$
$0.20$	20	$1.481$	$1.097$	$0.736$	$0.395$	$0.225$	$1.094$	$0.717$	$0.373$	$0.361$
$0.20$	50	$1.559$	$1.293$	$0.808$	$0.276$	$0.165$	$1.287$	$0.748$	$0.162$	$0.084$
$0.20$	100	$1.579$	$1.386$	$0.872$	$0.197$	$0.135$	$1.381$	$0.787$	$0.061$	$0.019$

Table 3. Empirical MSEs of the posterior means for $\mu$.

$\mathit{\epsilon }$	$\mathit{n}$	Bayes	${\mathit{R}}^{\left(\mathit{\alpha }\right)}$-Posterior				$\mathit{\gamma }$-Posterior
		$\mathit{\alpha },\mathit{\gamma }\to \mathbf{0.0}$	$\mathit{\alpha }=\mathbf{0.2}$	$\mathit{\alpha }=\mathbf{0.3}$	$\mathit{\alpha }=\mathbf{0.5}$	$\mathit{\alpha }=\mathbf{0.7}$	$\mathit{\gamma }=\mathbf{0.2}$	$\mathit{\gamma }=\mathbf{0.3}$	$\mathit{\gamma }=\mathbf{0.5}$	$\mathit{\gamma }=\mathbf{0.7}$
Uniform prior
$0.00$	20	$0.050$	$0.051$	$0.053$	$0.090$	$0.282$	$0.051$	$0.053$	$0.057$	$0.078$
$0.00$	50	$0.020$	$0.021$	$0.022$	$0.023$	$0.027$	$0.021$	$0.022$	$0.023$	$0.025$
$0.00$	100	$0.010$	$0.010$	$0.011$	$0.012$	$0.013$	$0.010$	$0.011$	$0.012$	$0.013$
$0.05$	20	$0.223$	$0.098$	$0.081$	$0.280$	$1.081$	$0.096$	$0.075$	$0.076$	$0.159$
$0.05$	50	$0.144$	$0.031$	$0.025$	$0.025$	$0.039$	$0.031$	$0.025$	$0.025$	$0.027$
$0.05$	100	$0.118$	$0.015$	$0.012$	$0.013$	$0.013$	$0.014$	$0.012$	$0.013$	$0.014$
$0.20$	20	$1.761$	$1.267$	$1.127$	$2.296$	$4.781$	$1.254$	$1.031$	$0.906$	$1.402$
$0.20$	50	$1.571$	$0.950$	$0.647$	$0.311$	$0.879$	$0.944$	$0.613$	$0.188$	$0.088$
$0.20$	100	$1.509$	$0.844$	$0.494$	$0.095$	$0.052$	$0.840$	$0.463$	$0.046$	$0.019$
Reference prior
$0.00$	20	$0.050$	$0.052$	$0.054$	$0.062$	$0.077$	$0.052$	$0.054$	$0.063$	$0.076$
$0.00$	50	$0.020$	$0.021$	$0.022$	$0.024$	$0.027$	$0.021$	$0.022$	$0.024$	$0.028$
$0.00$	100	$0.010$	$0.010$	$0.011$	$0.012$	$0.013$	$0.010$	$0.011$	$0.012$	$0.014$
$0.05$	20	$0.223$	$0.080$	$0.065$	$0.067$	$0.086$	$0.080$	$0.064$	$0.077$	$0.066$
$0.05$	50	$0.144$	$0.028$	$0.024$	$0.026$	$0.028$	$0.028$	$0.024$	$0.030$	$0.026$
$0.05$	100	$0.118$	$0.014$	$0.012$	$0.013$	$0.014$	$0.014$	$0.012$	$0.015$	$0.013$
$0.20$	20	$1.761$	$1.106$	$0.744$	$0.385$	$0.564$	$1.104$	$0.727$	$0.304$	$0.280$
$0.20$	50	$1.571$	$0.881$	$0.497$	$0.111$	$0.057$	$0.879$	$0.477$	$0.082$	$0.042$
$0.20$	100	$1.509$	$0.809$	$0.410$	$0.041$	$0.020$	$0.807$	$0.385$	$0.026$	$0.019$
Moment matching prior
$0.00$	20	$0.050$	$0.052$	$0.055$	$0.064$	$0.080$	$0.052$	$0.055$	$0.063$	$0.074$
$0.00$	50	$0.020$	$0.021$	$0.022$	$0.025$	$0.028$	$0.021$	$0.022$	$0.025$	$0.028$
$0.00$	100	$0.010$	$0.010$	$0.011$	$0.012$	$0.014$	$0.010$	$0.011$	$0.012$	$0.014$
$0.05$	20	$0.223$	$0.075$	$0.063$	$0.067$	$0.085$	$0.075$	$0.063$	$0.067$	$0.076$
$0.05$	50	$0.144$	$0.028$	$0.024$	$0.026$	$0.030$	$0.028$	$0.024$	$0.026$	$0.029$
$0.05$	100	$0.118$	$0.014$	$0.012$	$0.013$	$0.014$	$0.014$	$0.012$	$0.013$	$0.015$
$0.20$	20	$1.761$	$1.039$	$0.648$	$0.295$	$0.394$	$1.043$	$0.655$	$0.286$	$0.290$
$0.20$	50	$1.571$	$0.852$	$0.453$	$0.088$	$0.044$	$0.853$	$0.443$	$0.078$	$0.043$
$0.20$	100	$1.509$	$0.794$	$0.385$	$0.034$	$0.018$	$0.794$	$0.365$	$0.025$	$0.018$

Table 4. Empirical MSEs of the posterior means for $\sigma$.

$\mathit{\epsilon }$	$\mathit{n}$	Bayes	${\mathit{R}}^{\left(\mathit{\alpha }\right)}$-Posterior				$\mathit{\gamma }$-Posterior
		$\mathit{\alpha },\mathit{\gamma }\to \mathbf{0.0}$	$\mathit{\alpha }=\mathbf{0.2}$	$\mathit{\alpha }=\mathbf{0.3}$	$\mathit{\alpha }=\mathbf{0.5}$	$\mathit{\alpha }=\mathbf{0.7}$	$\mathit{\gamma }=\mathbf{0.2}$	$\mathit{\gamma }=\mathbf{0.3}$	$\mathit{\gamma }=\mathbf{0.5}$	$\mathit{\gamma }=\mathbf{0.7}$
Uniform prior
$0.00$	20	$0.033$	$0.062$	$0.104$	$1.110$	$8.455$	$0.057$	$0.080$	$0.195$	$0.747$
$0.00$	50	$0.011$	$0.015$	$0.019$	$0.034$	$0.161$	$0.015$	$0.017$	$0.025$	$0.036$
$0.00$	100	$0.005$	$0.006$	$0.007$	$0.011$	$0.018$	$0.006$	$0.007$	$0.009$	$0.012$
$0.05$	20	$0.761$	$0.424$	$0.471$	$7.528$	$37.358$	$0.379$	$0.309$	$0.673$	$3.370$
$0.05$	50	$0.553$	$0.095$	$0.051$	$0.066$	$0.950$	$0.087$	$0.040$	$0.035$	$0.047$
$0.05$	100	$0.482$	$0.039$	$0.017$	$0.018$	$0.031$	$0.035$	$0.014$	$0.012$	$0.014$
$0.20$	20	$3.262$	$4.185$	$5.830$	$55.081$	$138.264$	$3.874$	$4.117$	$8.080$	$29.181$
$0.20$	50	$2.816$	$2.706$	$2.229$	$1.895$	$27.513$	$2.638$	$1.962$	$0.741$	$0.454$
$0.20$	100	$2.682$	$2.405$	$1.704$	$0.483$	$0.506$	$2.372$	$1.526$	$0.146$	$0.038$
Reference prior
$0.00$	20	$0.027$	$0.030$	$0.033$	$0.040$	$0.059$	$0.030$	$0.032$	$0.041$	$0.058$
$0.00$	50	$0.010$	$0.011$	$0.012$	$0.015$	$0.017$	$0.011$	$0.012$	$0.015$	$0.020$
$0.00$	100	$0.005$	$0.006$	$0.006$	$0.007$	$0.008$	$0.006$	$0.006$	$0.007$	$0.009$
$0.05$	20	$0.611$	$0.153$	$0.083$	$0.068$	$0.101$	$0.145$	$0.073$	$0.050$	$0.054$
$0.05$	50	$0.504$	$0.054$	$0.023$	$0.019$	$0.021$	$0.051$	$0.021$	$0.017$	$0.021$
$0.05$	100	$0.459$	$0.027$	$0.011$	$0.009$	$0.010$	$0.025$	$0.010$	$0.008$	$0.010$
$0.20$	20	$2.731$	$2.283$	$1.624$	$0.941$	$0.982$	$2.232$	$1.482$	$0.548$	$0.304$
$0.20$	50	$2.633$	$2.165$	$1.330$	$0.341$	$0.215$	$2.140$	$1.218$	$0.171$	$0.048$
$0.20$	100	$2.595$	$2.158$	$1.268$	$0.144$	$0.070$	$2.143$	$1.144$	$0.046$	$0.014$
Moment matching prior
$0.00$	20	$0.026$	$0.028$	$0.031$	$0.040$	$0.063$	$0.028$	$0.032$	$0.042$	$0.054$
$0.00$	50	$0.010$	$0.011$	$0.012$	$0.015$	$0.019$	$0.011$	$0.012$	$0.015$	$0.020$
$0.00$	100	$0.005$	$0.006$	$0.006$	$0.007$	$0.009$	$0.006$	$0.006$	$0.007$	$0.009$
$0.05$	20	$0.525$	$0.105$	$0.058$	$0.046$	$0.052$	$0.104$	$0.057$	$0.048$	$0.056$
$0.05$	50	$0.470$	$0.043$	$0.020$	$0.017$	$0.018$	$0.041$	$0.019$	$0.017$	$0.021$
$0.05$	100	$0.443$	$0.023$	$0.010$	$0.008$	$0.009$	$0.022$	$0.009$	$0.008$	$0.010$
$0.20$	20	$2.411$	$1.809$	$1.132$	$0.441$	$0.186$	$1.816$	$1.137$	$0.461$	$0.385$
$0.20$	50	$2.507$	$1.974$	$1.120$	$0.222$	$0.082$	$1.971$	$1.065$	$0.153$	$0.054$
$0.20$	100	$2.532$	$2.065$	$1.148$	$0.106$	$0.040$	$2.059$	$1.054$	$0.043$	$0.015$

The simulation results of the empirical bias and MSE of posterior means of $\mu$ and $\sigma$ are provided by Table 1, Table 2, Table 3 and Table 4. We consider three prior distributions for $(\mu ,\sigma )$, namely uniform, reference, and moment matching priors. In these tables, we set $\nu =6$, $\epsilon =0.00,0.05,0.20$ and $n=20,50,100$. We also set the tuning parameters for the $R^{\left(\alpha \right)}$- and $\gamma$-posteriors as $0.2,0.3,0.5,0.7$.

Table 1 and Table 3 show the empirical bias and MSE of the posterior means of mean parameter $\mu$ based on the standard posterior and the $R^{\left(\alpha \right)}$- and $\gamma$-posteriors. The empirical bias and MSE for the two robust methods are smaller than those of the standard posterior mean (denoted by “Bayes” in Table 1, Table 2, Table 3 and Table 4) in the presence of outliers for a large sample size. When there are no outliers ($\epsilon =0$), it seems that the three methods are comparable. On the other hand, when $\epsilon =0.05$ and $\epsilon =0.20$, the standard posterior mean gets worse, while the performances of the posterior means based on the $R^{\left(\alpha \right)}$-posterior and the $\gamma$-posterior are comparable for both empirical bias and MSE.

We also present the results of the estimation for variance parameter $\sigma$ in Table 2 and Table 4. When there are no outliers, the performances of robust Bayes estimators under the uniform prior are slightly worse. On the other hand, the reference and moment matching priors provide relatively reasonable results even if the sample size is small and $\epsilon =0$. The empirical bias and MSE of the $R^{\left(\alpha \right)}$-posterior and the $\gamma$-posterior means for $\alpha ,\gamma =0.5,0.7$ remain small even if the contamination ratio $\epsilon$ is not small. In particular, the empirical bias and MSE of the $\gamma$-posterior means for $\sigma$ are shown to be drastically smaller than those of the $R^{\left(\alpha \right)}$-posterior.

Figure 1 shows the results of the empirical bias and MSE of the posterior means of $\mu$ and $\sigma$ under the uniform, reference, and moment matching priors when $\nu =6$ (fixed) and the contamination ratio $\epsilon$ varies from $0.00$ to $0.30$. In all cases, we can find that the standard posterior means (i.e., cases $\alpha ,\gamma =0$) do not work well. For the estimation of mean parameter $\mu$, the $R^{\left(\alpha \right)}$- and $\gamma$-posterior means seems to be reasonable for the value of $\epsilon$ between $0.0$ and $0.20$. In particular, the $\gamma$-posterior means under reference and moment matching priors have better performance even if $\epsilon =0.30$. For the estimation of variance parameter $\sigma$, the $R^{\left(\alpha \right)}$-posterior means under the uniform prior have larger bias and MSE than the other methods. The $\gamma$-posterior mean with $\gamma =1.0$ still may be better than other competitors for any $\epsilon \in [0,0.30]$. For $\alpha ,\gamma =0.5$, the $R^{\left(\alpha \right)}$- and $\gamma$-posterior means seem to be comparable.

Figure 1. The horizontal axis is the contamination ratio $\epsilon$. The red lines show the empirical bias and MSE of the $\gamma$-posterior means under the three priors when $n=100$ and $\nu =6$. Similarly, the blue and green lines show that of the $R^{\left(\alpha \right)}$-posterior and ordinary posterior means, respectively. The uniform, reference, and moment matching priors are denoted by “Uni”, “Ref”, and “MM”, respectively.

Figure 2 also presents the results of the empirical bias and MSE of the posterior means of $\mu$ and $\sigma$ under the same priors as Figure 1 when the contamination ratio is $\epsilon =0.20$ (fixed) and $\nu$ varies from $0.0$ to $10.0$. For the estimation of mean parameter $\mu$ in Figure 2, the empirical bias and MSE for the robust estimators seem to be nice regardless of $\nu$ except for the case of the $R^{\left(\alpha \right)}$-posterior under the uniform prior. Although we can find that some differences appear near $\nu =4$, the $\gamma$-posterior means with $\gamma =1.0$ have better performance for the estimation of both mean $\mu$ and variance $\sigma$ for all $\nu \in [0,10]$.

Figure 2. The horizontal axis is the location parameter $\nu$ of the contamination distribution. The red lines show the empirical bias and MSE of the $\gamma$-posterior means under the three priors when $n=100$ and $\epsilon =0.20$. Similarly, the blue and green lines show that of the $R^{\left(\alpha \right)}$-posterior and ordinary posterior means, respectively. The uniform, reference, and moment matching priors are denoted by “Uni”, “Ref”, and “MM”, respectively.

In these simulation studies, the $\gamma$-posterior mean under the reference and moment matching priors seems to have better performance for the joint estimation of $(\mu ,\sigma )$ in most scenarios. Although we provide the results for the univariate normal distribution, the other distribution (including the multivariate distribution) should be also considered in the future.

2

4.2. Selection of Tuning Parameters

The selection of a tuning parameter $\gamma$ (or $\alpha$) is very challenging, and to the best of our knowledge, there is no optimal choice of $\gamma$. The tuning parameter $\gamma$ controls the degree of robustness, that is, if we set large $\gamma$, we obtain higher robustness. However, there is a trade-off between the robustness and efficiency of estimators. One of the solutions for this problem is to use the asymptotic relative efficiency (ARE) (see, e.g., [11]). It should be noted that [11] only dealt with a one parameter case. In general, the asymptotic relative efficiency of the robust posterior mean ${\hat{\mathit{\theta }}}^{\left(\gamma \right)}$ of p-dimensional parameter $\mathit{\theta }$ relative to the usual posterior mean $\hat{\mathit{\theta }}$ is defined by:

$$\begin{array}{c}\hfill \mathrm{ARE}({\hat{\mathit{\theta }}}^{\left(\gamma \right)},\hat{\mathit{\theta }}):={\left(\frac{det\left(V\left(\mathit{\theta }\right)\right)}{det\left(V^{\left(\gamma \right)}\left(\mathit{\theta }\right)\right)}\right)}^{1/p}\end{array}$$

(see, e.g., [24]). This is the ratio of the determinants of the covariance matrices, raised to the power of $1/p$, where p is the dimension of the parameter $\mathit{\theta }$. We now calculate the $\mathrm{ARE}({\hat{\mathit{\theta }}}^{\left(\gamma \right)},\hat{\mathit{\theta }})$ in our simulation setting. After some calculations, the asymptotic relative efficiency is given by:

$$\begin{array}{c}\hfill \mathrm{ARE}({\hat{\mathit{\theta }}}^{\left(\gamma \right)},\hat{\mathit{\theta }})={\left(\frac{2}{{(1+\gamma )}^6(1+2\gamma )(2+4\gamma +3{\gamma }^2)}\right)}^{1/2}=:h\left(\gamma \right)\end{array}$$

for $\gamma >0$. We note that it holds $h\left(\gamma \right)\to 1$ as $\gamma \to 0$. Hence, we may be able to choose $\gamma$ to allow for the small inflation of the efficiency. For example, if we require the value of the asymptotic relative efficiency $\mathrm{ARE}=0.95$, we may choose the value of $\gamma$ as the solution of the equation $h\left(\gamma \right)=0.95$ (see Table 5). The curve of the function $h\left(\gamma \right)$ is also given in Figure 3. Several authors have provided methods for the selection of the tuning parameters (e.g., [25,26,27]). Reference [5] focused on the reduction of the latent bias of the estimator, and they recommended setting $\gamma =1$ for the normal mean-variance estimation problem; however, it seems to be unreasonable in terms of the asymptotic relative efficiency (see Table 5 and Figure 3). To the best of our knowledge, there is no method that is robust and efficient under the heavy contamination setting. Hence, other methods that have higher efficiency under heavy contamination should be considered in the future.

Table 5. The value of $\gamma$ and the corresponding asymptotic relative efficiency.

$\gamma$	$0.01$	$0.1$	$0.3$	$0.5$
ARE	0.951489	0.6222189	0.2731871	0.1359501

Figure 3. The curve of the asymptotic relative efficiency for normal mean and variance estimation under the $\gamma$-posterior.

5. Concluding Remarks

We consider objective priors for divergence-based robust Bayesian estimation. In particular, we prove that the reference and moment matching priors under quasi-posterior based on the $\gamma$-divergence are robust against unknown quantities in a data generating distribution. The performance of the corresponding posterior means is illustrated through some simulation studies. However, the proposed objective priors are often improper, and showing their posterior propriety remains as future research. Our results should be extended to other settings. For example, Kanamori and Fujisawa [28] proposed the estimation of the contamination ratio using an unnormalized model. Examining such a problem from the Bayesian perspective is also challenging because there is the problem of how to set a prior distribution for an unknown contamination ratio. Furthermore, it would also be interesting to consider an optimal data-dependent choice of tuning parameter $\gamma$.