Asymptotic Properties for Methods Combining the Minimum Hellinger Distance Estimate and the Bayesian Nonparametric Density Estimate

In frequentist inference, minimizing the Hellinger distance between a kernel density estimate and a parametric family produces estimators that are both robust to outliers and statistically efficient when the parametric family contains the data-generating distribution. This paper seeks to extend these results to the use of nonparametric Bayesian density estimators within disparity methods. We propose two estimators: one replaces the kernel density estimator with the expected posterior density using a random histogram prior; the other transforms the posterior over densities into a posterior over parameters through minimizing the Hellinger distance for each density. We show that it is possible to adapt the mathematical machinery of efficient influence functions from semiparametric models to demonstrate that both our estimators are efficient in the sense of achieving the Cramér-Rao lower bound. We further demonstrate a Bernstein-von-Mises result for our second estimator, indicating that its posterior is asymptotically Gaussian. In addition, the robustness properties of classical minimum Hellinger distance estimators continue to hold.


Introduction
This paper develops Bayesian analogs of minimum Hellinger distance methods. In particular, we aim to produce methods that enable a Bayesian analysis to be both robust to unusual values in the data and to retain their asymptotic precision when a proposed parametric model is correct.
All statistical models include assumptions which may or may not be true of the mechanisms producing a given data set. Robustness is a desired property in which a statistical procedure is relatively insensitive to deviations from these assumptions. For frequentist inference, concerns are largely associated with distributional robustness: the shape of the true underlying distribution deviates slightly from the assumed model. Usually, this deviation represents the situation where there are some outliers in the observed data set; see [1] for example. For Bayesian procedures, the deviations may come from the model, prior distribution, or utility function, or some combination thereof. Much of the literature on Bayesian robustness has been concerned with the prior distribution or utility function. By contrast, the focus of this paper is robustness with respect to outliers in a Bayesian context, a relatively understudied form of robustness for Bayesian models. For example, we know that Bayesian models with heavy tailed data distributions are robust with respect to outliers for the case of one single location parameter estimated by many observations. However, as a consequence of the Crámer-Rao lower bound and the efficiency of the MLE, modifying likelihoods to account for outliers will usually result in a loss of precision in parameter estimates when they are not necessary. The methods we propose, and the study of their robustness properties, will provide an alternative means of making any i.i.d. data distribution robust to outliers that do not lose efficiency when no outliers are present. We speculate that they can be extended beyond i.i.d. data as in [2], but we do not pursue this here.
Suppose we are given the task of estimating θ 0 ∈ Θ from independent and identically distributed univariate random variables X 1 , . . . , X n , where we assume each X i has density f θ 0 ∈ F = { f θ : θ ∈ Θ}. Within the frequentist literature, minimum Hellinger distance estimates proceed by first estimating a kernel densityĝ n (x) and then choosing θ to minimize the Hellinger distance h( f θ , g n ) n (x)} 2 dx] 1/2 . The minimum Hellinger distance estimator was shown in [3] to have the remarkable properties of being both robust to outliers and statistically efficient, in the sense of asymptotically attaining the information bound, when the data are generated from f θ 0 . These methods have been generalized to a class of minimum disparity estimators, based on alternative measures of the difference between a kernel density estimate and a parametric model, which have been studied since then, e.g., [4][5][6][7][8]. While some adaptive M-estimators can be shown to retain both robustness and efficiency, e.g., [9], minimum disparity methods are the only generic methods we are aware of that retain both properties and can also be readily employed within a Bayesian context. In this paper, we only consider Hellinger distance in order to simplify the mathematical exposition; the extension to more general disparity methods can be made following similar developments to those in [5,7].
Recent methodology proposed in [2] suggested the use of disparity-based methods within Bayesian inference via the construction of a "disparity likelihood" by replacing the likelihood function when calculating the Bayesian posterior distribution; they demonstrated that the resulting expected a posteriori estimators retain the frequentist properties studied above. These methods first obtain kernel density estimates from data and then calculate the disparity between the estimated density function and the corresponding density functions in the parametric family.
In this paper, we propose the use of Bayesian non-parametric methods instead of the classical kernel methods in applying the minimum Hellinger distance method. One method we proposed is just to replace the kernel density estimate used in classical minimum Hellinger distance estimate by the Bayesian nonparametric expected a posteriori density, which we denote by MHB (minimum Hellinger distance method using a Bayesian nonparametric density estimate). The second method combines the minimum Hellinger distance estimate with the Bayesian nonparametric posterior to give a posterior distribution of the parameter of interest. This latter method is our main focus. We show that it is more robust than usual Bayesian methods and demonstrate that it retains asymptotic efficiency, hence the precision of the estimate is maintained. So far as we are aware, this is the first Bayesian method that can be applied generically and retain both robustness and (asymptotic) efficiency. We denote it by BHM (Bayesian inference using a minimum Hellinger distance).
To study the properties of the proposed new methods, we treat both MHB and BMH as special cases of semi-parametric models. The general form of a semi-parametric model has a natural parametrization (θ, η) → P θ,η , where θ ∈ Θ is a Euclidean parameter and η ∈ H belongs to an infinite-dimensional set. For such models, θ is the parameter of primary interest, while η is a nuisance parameter. Asymptotic properties of some of Bayesian semi-parametric models have been discussed in [10]. Our disparity based methods involve parameters in Euclidean space and Hilbert space, with the former being of most interest. However, unlike many semi-parametric models in which P θ,η ∈ P is specified jointly by θ and η, in our case, the finite dimensional parameter and the nonparametric density functions are parallel specifications of the data distribution. Therefore, standard methods to study asymptotic properties of semi-parametric models will not apply to the study of disparity-based methods. Nevertheless, considering the problem of estimating ψ(P) of some function ψ : P → R d , where P is the space of the probability models P, semi-parametric models and disparity-based methods can be unified into one framework.
The MHB and BMH methods are introduced in detail in Section 2, where we also discuss some related concepts and results, such as tangent sets, information, consistency, and the specific nonparametric prior that we employ. In Section 3, both MHB and BMH are shown to be efficient, in the sense that asymptotically the variance of the estimate achieves the lower bound of the Cramér-Rao theorem. For MHB, we show that asymptotic normality of the estimate holds, where the asymptotic variance is the inverse of the Fisher information. For BMH, we show that the Bernstein von Mises (BvM) theorem holds. The robustness property and further discussion of these two methods are given in Sections 4 and 5, respectively. A broader discussion is given in Section 6.

Minimum Hellinger Distance Estimates
Assume that random variables X 1 , . . . , X n are independent and identically distributed (iid) with density belonging to a specified parametric family F = { f θ : θ ∈ Θ}, where all the f θ in the family have the same support, denoted by supp( f ). For simplicity, we use X n to denote the random variables X 1 , . . . , X n . More flexibly, we model X n ∼ g n , where g is a probability density function with respect to the Lebesgue measure on supp( f ). Let G denote the collection of all such probability density functions. If the parametric family contains the data-generating distribution, then g = f θ for some θ. Formally, we can denote the probability model of the observations in the form of a semi-parametric model (θ, g) → P θ,g . We aim at estimating θ and consider g as a nuisance parameter, which is typical of semi-parametric models.
Let π denote a prior on G , and for any measurable subset B ⊂ G , the posterior probability of g ∈ B given X n is Let g * n = gπ(dg | X n ) denote the Bayesian nonparametric expected a posteriori estimate. Our first proposed method can be described formally as follows: MHB: Minimum Hellinger distance estimator with Bayesian nonparametric density estimation: This estimator replaces the kernel density estimate in the classical minimum Hellinger distance method introduced in [3] by the posterior expectation of the density function. For this method, we will viewθ 1 as the value at g * n of a functional T : G → Θ, which is defined via where · denotes the L 2 metric. We can also writeθ 1 as T(g * n ). In a more general form, what we estimate is the value ψ(P) of some functional ψ : P → R d , where the P stands for the common distribution from which data are generated, and P is the set of all possible values of P, which also denotes the corresponding probability model. In the setting of minimum Hellinger distance estimation, the model P is set as F × G , P can be specified as P θ,g , and ψ(P) = ψ(P θ,g ) = θ. For the methods we proposed in this paper, we will focus on the functional T : G → Θ, for a given F , as defined above. Note that the constraint associated with the family F is implicitly applied by T.
Using functional T, we can also propose a Bayesian method, which assigns nonparametric prior on the density space and gives inference on the unknown parameter θ of a parametric family as follows: BMH: Bayesian inference with minimum Hellinger distance estimation: A nonparametric prior π on the space G and the observation X n leads to the posterior distribution π(g | X n ), which can then be converted to the posterior distribution of the parameter θ ∈ Θ through the functional T : G → Θ.
In the following subsections, we discuss properties associated with the functional T as well as the consistency of MHB and BHM, and we provide a detailed example of the random histogram prior that we will employ and its properties that will be used for the discussion of efficiency in Section 2.1.

Tangent Space and Information
In this subsection, we obtain the efficient influence function of the functional T on the linear span of the tangent set on g 0 and show that the local asymptotic normality (LAN) expansion related to the norm of the efficient influence function attains the Caramér-Rao bound. These results play important roles in showing that BvM holds for the BMH method in the next section.
Estimating the parameter by T(g) under the assumption g ∈ G uses less information than estimating this parameter for g ∈ G * ⊂ G . Hence, the lower bound of the variance of T(g) for g ∈ G should be at least the supremum of the lower bounds of all parametric sub-models G * = {G λ : λ ∈ Λ} ⊂ G .
To use mathematical tools such as functional analysis to study the properties of the proposed methods, we introduce some notations and concepts below. Without loss of generality, we consider one-dimensional sub-models G * , which pass through the "true" distribution, denoted by G 0 with density function g 0 . We say a sub-model indexed by t, {g t : 0 < t < } ⊂ G , is differentiable in quadratic mean at t = 0 if we have that, for some measurable function q : supp(g 0 ) → R, where G t is the cumulative distribution function associated with g t . Functions q(x)s are known as the score functions associated with each sub-model. The collection of these score functions, which is called a tangent set of the model G at g 0 and denoted byĠ g 0 , is induced by the collection of all sub-models that are differentiable at g 0 . We say that T is differentiable at g 0 relative to a given tangent setĠ g 0 , if there exists a continuous linear mapṪ g 0 : L 2 (G 0 ) → R such that for every q ∈Ġ g 0 and a sub-model t → g t with score function q, there is where L 2 (G 0 ) = {q : supp(g 0 ) → R, q 2 (x)g 0 (x)dx < ∞}. By the Riesz representation theorem for Hilbert spaces, the mapṪ g 0 can always be written in the form of an inner product with a fixed vector-valued, measurable functionT g 0 : supp(g 0 ) → R, LetT g 0 denote the unique function in linĠ g 0 , the closure of the linear span of the tangent set. The functionT g 0 is the efficient influence function and can be found as the projection of any other "influence function" onto the closed linear span of the tangent set.
For a sub-model t → g t whose score function is q, the Fisher information about t at 0 is G 0 q 2 = q 2 dG 0 . In this paper, we use the notation Fg to denote gdF for a general function g and distribution F. Therefore, the "optimal asymptotic variance" for estimating the functional t → T(g t ), evaluated at t = 0, is greater than or equal to the Caramér-Rao bound The supremum of the right-hand side (RHS) of the above expression over all elements of the tangent set is a lower bound for estimating T(g) given model G , if the true model is g 0 . The supremum can be expressed in the norm of the efficient influence functionT g 0 by Lemma 25.19 in [11]. The lemma and its proof is quite neat, and we reproduce it here for the completeness of the argument.

Lemma 1.
Suppose that the functional T : G → R is differentiable at g 0 relative to the tangent setĠ g 0 . Then Proof. This is a consequence of the Cauchy-Schwarz inequality (G 0Tg 0 q) 2 ≤ G 0T 2 g 0 G 0 q 2 and the fact that, by definition, the efficient influence function,T g 0 , is contained in the closure of linĠ G 0 . Now we show that functional T is differentiable under some mild conditions and construct its efficient influence function in the following theorem. (2), and for t ∈ Θ ⊂ R, let s t (x) denote f 1/2 θ (x) for θ = t. We assume that there existṡ t (x) ands t (x) both in L 2 , such that for α in a neighborhood of zero,

Theorem 1. For the functional T defined in Equation
where u α and v α converge to zero as α → 0. Assuming T(g 0 ) ∈ int(Θ), the efficient influence function of T is where a t converges to 0 as t → 0. In particular, for g 0 = f θ , Proof. Let the t-indexed sub-model be where q(x) satisfies q(x)g 0 (x)dx = 0 and q ∈ L 2 (g 0 ). By direct calculation, we see that q is the score function associated with such a sub-model at t = 0 in the sense of Equation (4) and thus the collection of q is the maximal tangent set. By the definition of T, T(g 0 ) maximizes s t (x)g 1/2 0 (x)dx. From Equation (6), we have that Since T(g 0 ) ∈ int(Θ), we have that Similarly, ṡ T(g t ) (x)g 1/2 t (x)dx = 0. Using Equation (7) to substituteṡ T(g t ) , we have that Since by the definition ofT, which requires T g 0 g 0 (x)dx = 0, we have that By the same argument we can show that, when g 0 = f θ , Equation (9) holds.
Some relatively accessible conditions under which Equations (6) and (7) hold are given by Lemmas 1 and 2 in [3]. We do not repeat them here. Now we can expand T at g 0 as whereT is given in Theorem 1 andr = 0.

Consistency of MHB and BMH
Since T(g) may have more than one value, the notation T(g) is used to denote any arbitrary one of the possible values. In [3], the existence, continuity in Hellinger distance, and uniqueness of functional T are ensured under the following condition: almost every When a Bayesian nonparametric density estimator is used, we assume the posterior consistency: A2 For any given > 0, π{g Under Conditions A1 and A2, consistency holds for MHB and BMH. Theorem 2. Suppose that Conditions A1 and A2 hold, then Note that the change of order of integration is due to Fubini's theorem and the last inequality is due to . Split the integral on the right-hand side of the above expression into two parts: The first term is bounded by by construction. By Condition A1, the posterior of measure of A c to 0 in probability as n → ∞. Since Hellinger distance is bounded by 2, so does the second term above. This completes the proof for g * 1/2 we need that the functional T is continuous and unique at f θ 0 , which is proved by Theorem 1 in [3] under Condition A1.
Part 2: By Condition A1 and Theorem 1 in [3], the functional T is continuous and unique at f θ 0 . Hence, for any given > 0, there exist δ > 0 such that It should be noted that, if we change the in Condition A2 to n , a sequence converging to 0, then we can apply the results for the concentration rate of the Bayesian nonparametric density estimation here. However, such an approach cannot lead to the general "efficiency" claim, no matter in the form of rate of concentration or asymptotic normality. There are two reasons for this. First, the rate of concentration for Bayesian nonparametric posterior is about n −2/5 for a rather general situation and (log n) a × n −1/2 , where a > 0, for some special cases (see [12][13][14]). This concentration rate is not sufficient in many situations to directly imply that the concentration of the corresponding parametric estimates achieves the lower bound of the variance given in the Cramér-Rao theorem. Second, the Hellinger distances between pairs of densities as functions of parameters vary among different parametric families. Therefore, obtaining the rate of concentration in parameters from the rate of convergence in the densities cannot be generally applied to different distribution families.
It should also be noted that, although Θ is required to be compact in Condition A1, Theorem 2 is useful for a Θ that is not compact, as long as the parametric family f θ : θ ∈ Θ can be re-parameterized where the space of new parameters can be embedded within a compact set. An example of re-parameterizing a general location-scale family with parameters µ ∈ R and σ ∈ R + to a family with parameters t 1 = tan −1 (µ) and t 2 = tan −1 (σ), where Θ (t 1 ,t 2 ) = (−π/2, π/2) × (0, π/2) and Θ ⊂Θ = [−π/2, π/2] × [0, π/2], is discussed in [3], and the conclusions of Theorem 1 in [3] is still valid for a location-scale family. Therefore, Theorem 2 remains valid for the same type of the families, whose parameter space may not be compact and for the same reasons; the compactness requirement stated in the theorem is mainly for mathematical simplicity.

Prior on Density Functions
We introduce a random histogram as an example for priors used in Bayesian nonparametric density estimation. It can be seen as a simplified version of a Dirichlet process mixture (DPM) prior, which is commonly used in practice. Both DPM and random histogram are mixture densities. While DPM uses a Dirichlet process to model the weights within an infinite mixture of kernels, the random histogram prior only has a finite number of components. Another difference is that, although we specify the form of the kernel function for DPM, the kernel function could be any density function in general, while the random histogram uses only the uniform density as its mixing kernel. Nevertheless, the limit on the finite number of the mixing components is not that important in practice, since the Dirichlet process will always be truncated in computation. In the next section, we will verify that the random histogram satisfies the conditions that are needed for our proposed methods to be efficient. On the other hand, although we believe that DPM should also lead to efficiency, the authors are unaware of the theoretical results or tools required to prove it. This is mostly due to the flexibility of DPM, which in turn significantly increases the mathematical complexity of the analysis.
For any k ∈ N, denote the set of all regular k bin histograms The set H k is a closed subset of L 2 [0, 1]. For any function f ∈ L 2 [0, 1], denote its projection in the We assign priors on H 1 k via k and (ω 1 , . . . , ω k ) for each k. A degenerate case is to let k = K n = o(n). Otherwise, let p k be a distribution on positive integers, where for all k large enough and some 0 < b 1 < b 2 < ∞. For example, Condition (13) is satisfied by the Poisson distribution, which is commonly used in Bayesian nonparametric models.
This means that the posterior of the density function concentrates around the projection g 0[k] of g 0 and around g 0 itself in terms of the Hellinger distance. We can easily conclude that π(K n | X n ) = 1 + o(1) from Equation (18) for g 0 ∈ C β . It should be noted that, although the priors we defined above are on the densities on [0, 1], this is for mathematical simplicity, which could easily be extended to the space of probability densities on any given compact set. Further, transformations of X n , similar to those discussed at the end of Section 2.2, can extend the analysis to the real line (refer to [3,16] for more example and details).

Efficiency
We say that both MHB and BMH methods are efficient if the lower bound of the variance of the estimate, in the sense of Cramèr and Rao's theorem, is achieved.

Asymptotic Normality of MHB
Consider the maximal tangent set at g 0 , which is defined as H T = {q ∈ L 2 (g 0 ), qg 0 = 0}. Denote the inner product on H T by q 1 , q 2 L = q 1 q 2 g 0 , which induces the L-norm as Note that the inner product ·, · L is equivalent to the inner product introduced in Section 2.1, and the induced L-norm corresponds to the local asymptotic normality (LAN) expansion. Refer to [17] and Theorem 25.14 in [11] for more details.
With functional T and priors on g defined in the previous section, Theorem 3 shows that the MHB method is efficient when the parametric family contains the true model. Theorem 3. Let two priors π 1 and π 2 be defined by Equations (13)- (14) and let a prior on k be either a Dirac mass at k = K n = n 1/2 (log n) −2 for π 1 or k ∼ π k given by Equation (13) for π 2 . Then the limit distribution of n 1/2 [T(g * n ) − T(g 0 )] under g 0 as n → ∞ is Norm(0, Proof. To prove this result, we verify Lemma 25.23 in [11], which is equivalent to showing that By the consistency result provided for priors π 1 and π 2 in the previous section, we consider only g * n ∈ A n,k for an n that is sufficiently large. Then by Equation (12) we have that will complete the proof. Due to 1 0 g 0 (x)T g 0 (x)dx = 0, we now need to show that 1 0 g * n (x)T g 0 (x)dx = (1/n) ∑ n i=1T g 0 (X i ) + o p (1). By the law of large numbers, we have that 1 n ∑ n i=1T g 0 (X i ) − G 0Tg 0 = o p (1), and 1 0 g * n (x)T g 0 (x)dx − G 0Tg 0 = o p (1) due to the posterior consistency demonstrated above. Therefore, we have that

The Bernstein von Mises Theorem for BMH
Theorem 2.1 in [15] yielded a general result and approach to show that the BvM Theorem holds for smooth functionals in some semi-parametric models. The theorem shows that, under the continuity and consistency condition, the moment generating function (MGF) of the parameter endowed with a posterior distribution can be calculated approximately through the local asymptotic normal (LAN) expansion, and its convergence to an MGF of some normal random variable can then be shown under some assumptions on the statistical model.
We will show that the BvM theorem holds for the BMH Method via Theorem 4. The result also shows that the approach given in [15] can be applied not only to simple examples but also to relatively complicated frameworks. To prove it, we introduced Lemma 2, which is modified from Proposition 1 in [15], the proof of which was not given explicitly in the original paper.
For mathematical simplicity, we assume that the true density f θ 0 belongs to the set F , which is restricted to the space of all densities that are bounded away from 0 and ∞ on [0, 1]. As noted above, the compactness of the domain can be relaxed by considering transformations of the parameters and random variables.
To state the lemma, we need several more notations. Assume that the functional T satisfies Equation (12) with bounded efficient influence functionT g 0 = 0. We denoteT g 0 byT, whereT [k] denotes the projection ofT on H k . For k ≥ 1, let and denote Lemma 2. Let g 0 belong to G , let the prior π be defined as in Section 2.3, and let Conditions (13,14,15) be satisfied. Consider estimating a functional T(g), differentiable with respect to the tangent set H T := {q ∈ L 2 (g 0 ), [0,1] qg 0 = 0} ⊂ H = L 2 (g 0 ), with efficient influence functionT g 0 bounded on [0, 1], and withr defined in Equation (12), for K n as introduced in Equation (15). If sup k∈K n sup g∈A n,k (M) √ nr(g, g 0 ) = o p (1) (24) for any M > 0 and A n,k (M) defined as in (16), as n → ∞, and then the BvM theorem for the functional T holds.
Proof. To show that BvM holds is to show that the posterior distribution converges to a normal distribution. If we have that then the proof will be completed by showing that the RHS of Equation (26) reduces from the mixture of normal to the target law N(0, V). By Condition (22), we have that V k goes to V uniformly for k ∈ K n . Due to the definition ofT and the Lemma 4 result (iii) in the supplement of [15], we have that By Conditions (25) and (23), the last line converges to 0 uniformly for k ∈ K n . Therefore, showing that for any given k, Equation (26) holds will complete the proof. We prove this by showing that the MGF (Laplace transformation) of the posterior distribution of the parameter of interest converges to the MGF of some normal distribution, which implies that the posterior converges to the normal distribution weakly by Lemmas 1 and 2 in supplement to [15] or Theorem 2.2 in [18].
First, consider the deterministic k = K n case. We calculate the MGF as A n e l n (g)−l n (g 0[k] ) dπ(g) (27) where l n (g) is the log-likelihood for given g and X n . Based on the LAN expansion of the log-likelihood and the smoothness of the functional, the exponent in the numerator on the RHS of the equation can be transformed with respect Note that G n = W n and add a term of (t 2 /2) T (k) 2 L . Re-arranging the RHS expression above, we have This is because the cross term in calculating the first term in the second line above is equal to the inner product term in the equation above it.
, the RHS of the above equation can be written as Substituting the corresponding terms on the RHS of Equation (27) by (28), we have that Notice that the integration in the denominator of the second term is an expectation based on a Dirichlet distribution on ω as described in Equation (14) and that g t,k = k ∑ k j=1 ζ j 1l I j , where with γ j = e tT j / √ n andT j := k I jT (k) . Let S γ −1 (ω) = ∑ k j=1 ω j γ −1 j , by (30). We then have S −1 γ (ζ) = S γ −1 (ω). Now using these notations, where ∆ ζ = S −k γ (ζ) ∏ k j=1 γ j is the Jacobian of the change of variable, (ω 1 , . . . , ω k−1 ) → (ζ 1 , . . . , ζ k−1 ), which is given in Lemma 5 in supplement of [15], and B(α Notice that, over the set A n,k , by Taylor's expansion. Expression (32) converges to 1 under Condition (15), so Expression (31) converges to Choosing M, such that Expression (33) equals 1 + o p (1). Notice that T (k) L,k = T [k] L . We then have that which completes the proof for a fixed k case. For a random k case, the proof will follow the same steps as the corresponding part in the proof for Theorem 4.2 in [15]. For completeness, we briefly sketch the proof here. Since k is not fixed, we of {1, 2, . . . , n/ log 2 n} such that π(K n | X n ) = 1 + o p (1) by the concentration property (a) of the random histogram, we have that π[B n | X n ] = 1 + o p (1). We rewrite the left-hand side (LHS) of , which is also equal to e t 2 T [k] 2 L (1 + o p (1)). Notice that o(1) in this expression is uniform in k. This is because it holds in the proof for a deterministic case for any given k < n. Therefore, Using Equations (23) and (25) together with the continuous mapping theorem for the exponential function yields that the last display converges in probability to e t 2 V/2 as n → ∞, which completes the proof.
The following theorem shows that Method 2 is efficient, the proof of which consists in verifying that the conditions in the above lemma are satisfied. Theorem 4. Suppose g 0 ∈ C β with β > 0. Let the prior on k be either a Dirac mass at k = K n = n 1/2 (log n) −2 or k ∼ π k given by (13), and let two priors π 1 and π 2 be defined by (14) and satisfy (13). Then, for all β > 1/2, the BvM holds for T( f ) for both π 1 and π 2 .
Proof. For T( f ) such that Equation (12) is satisfied, Condition (24) is satisfied obviously.
For Equation (23), the empirical process G n (T [k] −T) is controlled and will converge to 0 by applying Lemma 19.33 in [11].
Condition (25) is satisfied by Lemma 3 below. Now we show that Equation (22) holds: The last equality is based on Conclusion (3) in Lemma 4 in [19], and the last inequality is due to the assumption thatT is bounded. Then the last term is controlled by h( f , f n ), which completes the proof.
, under the deterministic k-prior with k = K n = n 1/2 (log n) −2 and β > 1/2, For the random k-prior, since we restrict g to be bounded from above and below, so the Hellinger and L 2 -distances considered are comparable. For a given k ∈ K n , by definition, there exists g * k ∈ H 1 k with h(g 0 , g * k ) ≤ M n (β), so which completes the proof.

Robustness Properties
In frequentist analysis, robustness is usually measured by the influence function and breakdown point of estimators. These have been used to study robustness in minimum Hellinger distance estimators in [3] and in more general minimum disparity estimators in [2,7].
In Bayesian inference, robustness is labeled "outlier rejection" and is studied under the framework of the "theory of conflict resolution". There is a large literature on this topic, e.g., [20][21][22]. While the results of [22] are only about symmetric distributions, [23] provides corresponding results covering a wider class of distributions with tails in the general exponential power family. These results provide a complete theory for the case of many observations and a single location parameter.
We examine the behavior of the methods MHB and BMH under a mixture model for gross errors. Let δ z denote the uniform density of the interval (z − , z + ), where > 0 is small, and let f θ,α,z = (1 − α) f θ + αδ z , where θ ∈ Θ and α ∈ [0, 1)] and z is a real number. The density f θ,α,z models a situation, where 100(1 − α)% observations are distributed from f θ , and 100α% of the observations are the gross errors located near z.
Theorem 5. For every α ∈ (0, 1) and every θ ∈ Θ, denote the mixture model for gross errors by f θ,α,z . We then have that lim z→∞ lim n→∞ T(g * n ) = θ, under the assumptions of Theorem 3 and that, for the BMH method, π(T(g) | X n ) → φ(θ, T f θ,α,z 2 L ) in the distribution as n → ∞ and z → ∞, where φ denotes the probability function of the normal distribution, when conditions in Theorem 4 are satisfied.
Proof. By Theorem 7 in [3], for functional T, as we defined and under the conditions in this theorem, we have that lim z→∞ T( f θ,α,z ) = θ.

Demonstration
We provide a demonstration of both BMH and MHB methods on two data sets: the classical Newcomb light speed data (see [24,25]), in which 2 out of 66 values are clearly negative oultiers, and a bivariate simulation containing 10% contamination in two asymmetric locations.
We have implemented the BMH and MHB methods using two Bayesian nonparametric priors: 1. the random histogram prior studied in this paper based on a fixed k = 100 with the range naturally extended to the range of the observed data (this is applied only to our first univariate example). 2. the popular Dirichlet Process (DP) kernel mixture of the form where the baseline distribution is the conjugate normal-inverted Wishart, Note that, when y i values are univariate observations, the inverse Wishart (IW) distribution reverts to an inverse Gamma distribution. To complete the model specification, independent hyperpriors are assumed We obtain posteriors for both using BUGS. We have elected to use BUGS here as opposed to the package DPpackage within R despite the latter's rather efficient MCMC algorithms because our BMH method requires direct access to samples from the posterior distribution as opposed to the expected a posteriori estimate. The R package distrEx is then used to construct the sampled density functions and calculated the Hellinger distance between the sampled densities from the nonparametric model and the assumed normal distribution. The R package optimx is also used to find the minima of the Hellinger distances. The time cost of our methods are dominated by the optimization step rather than by the obtaining of samples from the posterior density. We first apply BMH and MHB on the Simon Newcomb's measurements to measure the speed of light. The data contains 66 observations. For this example, we specify the parameters and hyper-parameters of the DPM as α = 1, m 2 = 0, s 2 = 1000, τ 1 = 1, τ 2 = 100, and ν 2 = 2, ψ 2 = 1. We plot the data and a bivariate contour of the BMH posterior for both the mean and variance of the assumed normal in Figure 1, where, despite outliers, the BvM result is readily apparent. Table 1 summarizes these estimates. We report the estimated mean and variance with and without the obvious outliers as well as the same quantities estimated using both MHB and BMH methods with the last of these being the expected a posteriori estimates. Quantities in parentheses given the "natural" standard error for each quantity: likelihood estimates correspond to standard normal theory-dividing the estimated standard error by √ n, and BMH standard errors are obtained from the posterior distribution. For MHB, we used a bootstrap and note that, while the computational cost involved in estimating MHB is significantly lower than BMH when obtaining a point estimate, the standard errors require and MCMC chain for each bootstrap, significantly raising the cost of obtaining these estimates. We observe that both prior specifications result in parameter estimates that are identical to two decimal places and very close to those obtained after removing outliers.  To examine the practical implementation of methods that go beyond our theoretical results, we applied these methods to a simulated two-dimensional data set of 100 data points generated from a standard normal with two contamination distributions. Specifically, our data distribution comes from 9 10 N where exactly five points were generated from each of the second-two Gaussians. Our DP prior used the same hyper-parameters as above with the exception that Ψ 1 was obtained from the empirical variance of the (contaminated) data, and (m 2 , S 2 ) were extended to their 2-dimensional form as (0, 0) T , diag(1000, 1000) . Figure 2 plots these data along with the posterior for the two means. Figure 3 provides posterior distributions for the components of the variance matrix. Table 2 presents estimation results for the full data and those with the contaminating distributions removed as well as from the BMH method. Here we again observe that BMH yields results that are very close to those obtained using the uncontaminated data. There is some more irregularity in our estimates, particularly in Figure 3, which we speculate is due to poor optimization. There is considerable scope to improve the numerics of minimum Hellinger distance methods more generally, but this is beyond the scope of this paper.

Discussion
This paper investigates the use of minimum Hellinger distance methods that replace kernel density estimates with Bayesian nonparametric models. We show that simply substituting the expected a posteriori estimator will reproduce the efficiency and robustness properties of the classical disparity methods first derived in [3]. Further, inducing a posterior distribution on θ through the posterior for g results in a Bernstein von Mises theorem and a distributional robustness result.
There are multiple potential extensions of this work. While we have focused on the specific pairing of Hellinger distance and random histogram priors, both of these can be generalized. A more general class of disparities was examined in [7], and we believe the extension of our methods to this class are straightforward. More general Bayesian nonparametric priors are discussed in [14], where the Dirichlet process prior has been particularly popular. Extensions to each of these priors will require separate analysis (e.g. [26]). Extensions of disparities to regression models were examined in [27] using a conditional density estimate, where equivalent Bayesian nonparametrics are not as well developed. Other modeling domains such as time series may require multivariate density estimates, resulting in further challenges.
Our results are a counterpoint to the Bayesian extensions of Hellinger distance methods in [2] where the kernel density was retained for g n but a prior was given for θ and the disparity treated as a log likelihood. Combining both these approaches represents a fully Bayesian implementation of disparity methods and is an important direction of future research.

Conflicts of Interest:
The authors declare no conflict of interest.