On Minimum Bregman Divergence Inference

Abstract

The density power divergence (DPD) is a well-studied member of the Bregman divergence family and forms the basis of widely used minimum divergence estimators that balance efficiency and robustness. In this paper, we introduce and study a new sub-class of Bregman divergences, termed the exponentially weighted divergence (EWD), designed to generate competitive and practically interpretable inference procedures. The EWD is constructed so that its associated weight function remains bounded within the interval $[0, 1]$, which facilitates a transparent interpretation of robustness through controlled downweighting of low-density observations and avoids excessive influence from high-density points. We develop minimum EWD estimators (MEWDEs) within a general framework accommodating independent but non-homogeneous data, thereby extending classical minimum divergence theory beyond the i.i.d. setting. Under standard regularity conditions, we establish Fisher consistency and asymptotic normality, and we analyze robustness properties through influence function calculations. The EWD framework is further extended to parametric hypothesis testing, for which we derive the asymptotic null distribution of a Bregman divergence-based test statistic. Extensive simulation studies and real-data applications demonstrate that the proposed estimators perform comparably to, and often more robustly than, existing DPD-based procedures, particularly under moderate to heavy contamination, while retaining high efficiency under clean data. Overall, the EWD provides a tractable and interpretable alternative within the Bregman divergence class for robust parametric estimation and testing.

1. Introduction

  Density-based minimum divergence methods are popular tools in statistical inference. In parametric estimation, this amounts to choosing the model density closest (in terms of the selected divergence) to the empirical data density. This approach often combines strong robustness properties with high asymptotic efficiency. An important class of density-based divergences is the class of $\varphi$ divergences (see [1]). Under standard regularity conditions, all minimum $\varphi$ divergence estimators have full asymptotic efficiency at the model [2]; many also have attractive robustness properties. The seminal Hellinger distance study of [3] appears to be the first which demonstrated that strong robustness properties may be achieved simultaneously with full asymptotic efficiency. Later, the same has been demonstrated with respect to much of the $\varphi$ divergence class (see, e.g., [4]). The usefulness of the corresponding procedures in providing robust alternatives to the likelihood ratio test has also been explored in the literature [2,4,5]. The approach has been further refined and extended in many directions by later authors. On the whole, the utility of the minimum divergence procedures based on $\varphi$ divergences is well established in the literature.

One major criticism of this inference procedure is that it inevitably involves the use of some form of non-parametric smoothing (such as kernel density estimation) to produce a continuous estimate of the true density. This can throw up several potential difficulties including the problematic bandwidth selection issue and the slow convergence of the of kernel density estimator to the “truth” (particularly for high dimensional data). The theoretical derivations are also harder. Development of methods which eliminate these difficulties may be worthwhile even if that involves a marginal loss in asymptotic efficiency.

An alternative class of minimum divergence estimators which avoids non-parametric smoothing in the construction of the empirical divergence is the class of minimum Bregman divergence estimators. An important example is the family of density power divergences (henceforth DPD($\alpha$), where $\alpha$ is the tuning parameter); the corresponding minimum density power divergence estimators (henceforth MDPDE($\alpha$)) have been shown to combine strong robustness properties with high asymptotic efficiency (see [6]). Divergences within the Bregman class have been called decomposable divergences by [7] and non-kernel divergences by [8]. These divergences have simple estimating equations and much of their asymptotic properties can be obtained from the M-estimation theory. The Kullback–Leibler divergence, which is a decomposable divergence, is the only common member between the $\varphi$ divergence and the Bregman divergence classes.

The application of Bregman divergences continues to expand rapidly into complex domains. Recent studies have successfully adapted these measures for multivariate time series analysis, including change-point and anomaly detection [9,10]. In the realm of geometric data analysis, variants like the Total Bregman Divergence have been applied to K-Means clustering for point cloud denoising [11]. On the theoretical front, new bounds on excess minimum risk have been established for generalized divergence measures [12], while recent work in econometrics has utilized these concepts for the robust learning of tail dependence [13].

In the context of density-based minimum divergence estimation, we have several “good” choices available. To justify the development of another family of estimators, one must demonstrate that the new estimators are competitive, if not better than the existing standard. Within the class of minimum divergence estimators which do not require any nonparametric smoothing, the MDPDE($\alpha$) is the current standard. In this paper, we will develop a family of divergences yielding minimum divergence estimators which satisfy this requirement; at the least, this family provides a highly competitive standard. Our proposed class of divergences will be called the exponentially weighted divergence family, indexed by a tuning parameter $\beta$ (henceforth referred to as EWD($\beta$)). The corresponding minimum divergence estimator will be denoted by MEWDE($\beta$). This divergence will also be useful in the field of hypothesis testing. Although the likelihood ratio test has several asymptotic optimality properties at the model, it is also known to have very poor robustness properties. Many density-based minimum distance procedures yield robust tests of hypothesis with high efficiency, e.g., Refs. [14,15,16]. Some of these papers address the general problem of parametric hypothesis testing based on the density power divergence. The present work considers this problem in the context of a general Bregman divergence, with special emphasis on our proposed EWD($\beta$) class. To summarize, the main results of this paper are fourfold:

• We introduce the exponentially weighted divergence (EWD), a novel sub-class of Bregman divergences. Unlike the popular density power divergence (DPD), the EWD is constructed to ensure the associated weight function remains bounded within the $[0, 1]$ interval.
• We derive the asymptotic properties (consistency and normality) of the minimum EWD estimator (MEWDE) for independent non-homogeneous data.
• We extend the framework to parametric hypothesis testing and derive the asymptotic null distribution of the test statistic.
• We evaluate the method through simulations and real-data applications, comparing its performance against the density power divergence (DPD) and classical robust estimators.

It is instructive to situate the proposed EWD within the broader landscape of robust statistics, while classical robust methods such as M-estimators, Least Median of Squares (LMS), and MM-estimators focus on minimizing residual functions, the proposed EWD approach belongs to the class of minimum divergence estimators. This distinction is crucial: by minimizing a probability distance, the MEWDE naturally achieves high asymptotic efficiency at the model—a property that often requires complex, multi-step procedures in standard high-breakdown regression (e.g., MM-estimation). Furthermore, within the divergence framework, it is important to distinguish the EWD from the class of $\varphi$-divergences [1], which includes the Hellinger distance and Pearson’s ${\chi }^2$; while $\varphi$-divergences offer strong geometric properties, they often result in estimating equations that are computationally intensive or require non-parametric density estimation components. In contrast, the EWD belongs to the Bregman divergence class. This ensures that the resulting estimators share the tractable, decomposable structure of the Maximum Likelihood Estimator (MLE) and can be easily implemented for any member of the exponential family. The EWD thus combines the desirable bounded-influence properties often sought in $\varphi$-divergence inference with the computational simplicity of the Bregman framework.

The rest of this paper is organized as follows. Section 2 introduces the general framework of minimum Bregman divergence estimation for independent non-homogeneous observations and defines our proposed sub-class, the EWD family. Section 3 establishes the theoretical properties of the estimator, including Fisher consistency and asymptotic normality, and analyzes its robustness via the influence function. Section 4 presents extensive simulation studies comparing the finite-sample performance of the proposed method against existing standards and discusses a data-driven strategy for selecting the optimal tuning parameter controlling trade-off between efficiency and robustness of MEWDEs. Section 5 demonstrates the practical utility of the estimator through several real-data examples, ranging from univariate problems to multiple linear regression. Section 6 extends the framework to robust hypothesis testing and derives the asymptotic null distribution of the proposed test statistic. Finally, Section 7 offers concluding remarks, while proofs and additional regression examples are provided in the Appendices.

2. The Bregman Divergence

2.1. Introduction to Minimum Bregman Divergence Estimation

The Bregman divergence between two densities g and f with common support $\mathcal{X}$ is defined as

$$D_B(g,f)={\int }_{\mathcal{X}}[B\left(g\left(x\right)\right)-B\left(f\left(x\right)\right)-(g\left(x\right)-f\left(x\right))B^{\prime }\left(f\left(x\right)\right)]dx,$$

(1)

where $B:\mathbb{R}\to \mathbb{R}$ is a strictly convex function and $B^{\prime }(·)$ is the derivative of B with respect to its argument. The integral in (1) is defined over the support $\mathcal{X}$ of the densities f and g. The strict convexity of B assures that the measure $D_B(g,f)$ is non-negative, and equals zero if and only if the arguments are identically equal. Ref. [17] discusses this and other measures in greater detail. We note that the convex functions $B\left(y\right)$ and $B^{*}\left(y\right)=B\left(y\right)+ay+c$ generate identical divergences in Equation (1) for $a,c\in \mathbb{R}$. In this section, we discuss minimum Bregman divergence inference in case of data that are independent, but not necessarily identically distributed. The i.i.d. data case emerges as a special case of this setup.

We assume that the data $X_1,\dots ,X_n$ are independent. For $i=1,2,\dots ,n$ we have $X_i\sim g_i$, where $g_i$ are possibly different densities with respect to some common dominating measure. We model $g_i$ by the family ${\mathcal{F}}_{i,\mathit{\theta }}=\{f_{i,\mathit{\theta }}\mid \mathit{\theta }\in \Omega \subseteq {\mathbb{R}}^s\}$ for each $i=1,2,\dots ,n$, where s denotes the dimension of the parameter vector $\mathit{\theta }$. An estimate of the Bregman divergence between the density $g_i$ and the associated model density $f_{i,\mathit{\theta }}(\in {\mathcal{F}}_{i,\mathit{\theta }})$ is given by $D_B({\widehat{g}}_i,f_{i,\mathit{\theta }})$, where ${\widehat{g}}_i$ is a non-parametric density estimate of $g_i$.

Since our aim is to reach some “common” value of $\mathit{\theta }$ (if it exists) which can be used to model each $g_i$ individually, it is intuitive to minimize the average divergence between the data points and the models, given by $H_n\left(\mathit{\theta }\right)=n^{-1}{\sum }_{i=1}^n{D}_B({\widehat{g}}_i,f_{i,\mathit{\theta }})$ with respect to $\mathit{\theta }$. In the presence of only one data point $X_i$ from density $g_i$, the best density estimate of $g_i$ is the (degenerate) density which puts the entire mass on the observed value of $X_i$ and this yields the objective function (ignoring the terms independent of $\mathit{\theta }$) given by

$$\begin{array}{c}\hfill H_n\left(\mathit{\theta }\right)={\displaystyle \frac{1}{n}}\sum _{i=1}^n\left[\int \left(f_{i,\mathit{\theta }}\left(x\right)B^{\prime }\left(f_{i,\mathit{\theta }}\left(x\right)\right)-B\left(f_{i,\mathit{\theta }}\left(x\right)\right)\right)dx-B^{\prime }\left(f_{i,\mathit{\theta }}\left(X_i\right)\right)\right]={\displaystyle \frac{1}{n}}\sum _{i=1}^n{V}_{i,\mathit{\theta }}\left(X_i\right).\end{array}$$

(2)

Let ${\nabla }_{\mathit{\theta }}$ represent the gradient with respect to $\mathit{\theta }$. Considering partial derivatives of the objective function in Equation (2), we arrive at the estimating equation

$$\begin{array}{c}\hfill \sum _{i=1}^n\left[u_{i,\mathit{\theta }}\left(X_i\right)w\left(f_{i,\mathit{\theta }}\left(X_i\right)\right)-\int u_{i,\mathit{\theta }}\left(x\right)w\left(f_{i,\mathit{\theta }}\left(x\right)\right)f_{i,\mathit{\theta }}\left(x\right)dx\right]=\mathbf{0},\end{array}$$

(3)

where $u_i\left(x\right)={\nabla }_{\mathit{\theta }}log\left(f_{i,\mathit{\theta }}\left(x\right)\right)$ is the likelihood score function of the density $f_{i,\mathit{\theta }}\left(x\right)$ used to model the i-th data point, and $w\left(t\right)=B^{″}\left(t\right)\times t$. If the data were i.i.d. with common density g, we would choose a common model density $f_{\mathbf{\theta }}$ for modeling and inference. Consequently, Equation (3) would yield

$$\sum _{i=1}^n\left[u_{\mathit{\theta }}\left(X_i\right)w\left(f_{\mathit{\theta }}\left(X_i\right)\right)-\int u_{\mathit{\theta }}\left(x\right)w\left(f_{\mathit{\theta }}\left(x\right)\right)f_{\mathit{\theta }}\left(x\right)dx\right]=\mathbf{0},$$

where $u_{\mathit{\theta }}\left(x\right)={\nabla }_{\mathit{\theta }}log\left(f_{\mathit{\theta }}\left(x\right)\right)$ is the likelihood score function of the model density $f_{\mathit{\theta }}\left(x\right)$. Equation (3) is a generalization of weighted likelihood estimating equation for the case of independent and non-homogeneous data.

2.2. The Exponentially Weighted Divergence

To develop new estimation procedures based on Bregman divergences, one can either (a) start with a specific convex function B as given in Equation (2) and construct a weighted likelihood equation, or (b) begin with a suitable robust weight representation as in Equation (3) and backtrack to recover the corresponding convex function B. We take the latter approach.

Philosophically, our treatment of outliers is probabilistic; an outlying point is one which has a small probability of occurrence under the model $f_{\mathit{\theta }}\in {\mathcal{F}}_{\mathit{\theta }}.$ We downweight those observations in the estimating equation for which the value of $f_{\mathit{\theta }}\left(x\right)$ is small. We plot the weight functions for some members of the DPD($\alpha$) family at different values of $\alpha$ in Figure 1. We note that the strength of downweighting increases with increasing $\alpha$. For $f_{\mathit{\theta }}\left(x\right)>1$, these weights grow unboundedly for all $\alpha >0$ as the argument increases.

As an alternative, we propose a new class of divergences based on a different choice of the weight function given by (depending on a non-negative index $\beta$)

$$w_{\beta }\left(t\right)=\left\{\begin{array}{cc}1-exp(-t/\beta )\hfill & \mathrm{if}\beta >0,\hfill \\ 1\hfill & \mathrm{if}\beta =0.\hfill \end{array}\right.$$

(4)

These weights smoothly drop to zero for decreasing values of the pdf $f_{\mathit{\theta }}\left(x\right)$ for $\beta >0$. Unlike the DPD($\alpha$) weights, they are bounded above by 1. We plot the weight functions given by Equation (4) for specific values of $\beta$ in Figure 2.

The likelihood equation may be recovered at $\beta =0$, where, to avoid the complications of division by zero, the weights have been defined by the corresponding limiting case as $\beta \to 0$. Using the relation $w\left(t\right)=B^{″}\left(t\right)\times t$, we recover the associated B function

$$B_{\beta }\left(x\right)={\displaystyle \frac{x^2}{\beta }}\left[\sum _{n=0}^{\infty }{\displaystyle \frac{{(-x/\beta )}^n}{(n+2)!(n+1)}}\right].$$

In the appendix, we show that this can be further simplified to

$$B_{\beta }\left(x\right)=-x+\gamma x+\beta -\beta exp(-x/\beta )+x\Gamma (0,x/\beta )+xlog(x/\beta ),$$

where $\gamma$ is the Euler–Mascheroni constant and $\Gamma (\alpha ,\beta )$ is the incomplete Gamma integral. We can equivalently consider the following simplified form of the defining function:

$$B_{\beta }^{*}\left(x\right)=-\beta exp(-x/\beta )+x\Gamma (0,x/\beta )+xlog(x/\beta ).$$

(5)

The associated Bregman divergence (which we will refer to as the exponentially weighted divergence EWD($\beta )$) has the form

$${\displaystyle \frac{1}{n}}\sum _{i=1}^n\left[\int \left(f_{i,\mathit{\theta }}\left(x\right)B^{\prime }\left(f_{i,\mathit{\theta }}\left(x\right)\right)-B\left(f_{i,\mathit{\theta }}\left(x\right)\right)\right)dx-B^{\prime }\left(f_{i,\mathit{\theta }}\left(X_i\right)\right)\right],$$

(6)

where $B=B_{\beta }^{*}$ is as in Equation (5). The MEWDE($\beta$) solves the estimating equation

$$\begin{array}{c}\hfill \sum _{i=1}^n\left[u_{i,\mathit{\theta }}\left(X_i\right)\left[1-exp\left(-f_{i,\mathit{\theta }}\left(X_i\right)/\beta \right)\right]-\int u_{i,\mathit{\theta }}\left(x\right)\left[1-exp\left(-f_{i,\mathit{\theta }}\left(x\right)/\beta \right)\right]f_{i,\mathit{\theta }}\left(x\right)dx\right]=\mathbf{0}.\end{array}$$

(7)

Remark 1.

We do not claim that the exponential weight function defining the EWD is unique among bounded weighting schemes within the Bregman divergence class. Indeed, infinitely many bounded weight functions could be constructed that yield valid Bregman divergences. The contribution of the present work is not to establish a universal optimality or uniqueness result, but rather to identify a particularly tractable and interpretable member of this class. The exponential form $w\left(t\right)=1-exp(-t/\beta )$ offers three practical advantages simultaneously: (i) weights are strictly bounded in $w\in [0,1]$, preventing inlier super-weighting; (ii) the resulting divergence retains the decomposable structure required for fast M-estimation; and (iii) the tuning parameter β admits a transparent probabilistic interpretation in terms of downweighting low-likelihood observations. These properties together motivate the EWD as a natural and practically appealing alternative to existing Bregman divergences such as the DPD.

Remark 2.

We note that the standard Maximum Likelihood Estimator (MLE) corresponds to the limiting case of the proposed EWD family. As the tuning parameter $\beta \to \infty$, the weight function converges to unity ($w\left(f\right(x\left)\right)\to 1$) for all x. Consequently, the estimating Equation (3) reduces to the classical likelihood score equation. This theoretical connection justifies the use of the MLE as the baseline for efficiency comparisons ($\beta \to \infty$) throughout the simulation and application sections.

3. Properties

3.1. Fisher Consistency of Minimum Bregman Divergence Estimators

We consider independent data $X_1,\dots ,X_n$ with $X_i\sim g_i$, where $g_i$ are (possibly) different densities with respect to some common dominating measure. Let $G_i$ be the distribution function associated with $g_i$ for $i=1,2,\dots ,n.$ The minimum Bregman divergence functional $T_B(G_1,G_2,\dots ,G_n)$ for non-homogeneous observations is given by the relation $T_B(G_1,G_2,\dots ,G_n)=\underset{\mathit{\theta }\in \Omega }{\mathrm{argmin}}{H}_n^{*}\left(\mathit{\theta }\right)$, where $H_n^{*}\left(\theta \right)={\displaystyle \frac{1}{n}}{\sum }_{i=1}^n{D}_B(g_i,f_{i,\theta })$ is the theoretical version of the expression as defined by Equation (2). As the Bregman divergence so defined is a genuine divergence in the sense that it is non-negative and attains its minimum if and only if each data generating distribution $G_i$ equals the model counterpart $F_{i,\theta }$, the functional T is Fisher consistent in the sense $T(F_{1,\theta },F_{2,\theta },\dots ,F_{n,\theta })=\mathit{\theta }$.

3.2. Asymptotic Properties

We derive the asymptotic distribution of the minimum Bregman divergence estimator ${\widehat{\mathit{\theta }}}_n$ defined by the relation ${\widehat{\mathit{\theta }}}_n=\underset{\mathit{\theta }\in \Omega }{\mathrm{argmin}}{H}_n\left(\mathit{\theta }\right)$, provided such a minimum exists, where $H_n\left(\mathit{\theta }\right)$ is as defined in Equation (2). In particular the presented results hold for MEWDE($\beta$).

We assume that the data are generated from the setup described in Section 3.1. We model $g_i$ by the parametric family ${\mathcal{F}}_{i,\mathit{\theta }}=\{f_{i,\mathit{\theta }}:\mathit{\theta }\in \Omega ,\Omega \subseteq {\mathbb{R}}^s\}$ for each $i=1,2,\dots ,n$. We also assume that there exists a best fitting value of $\mathit{\theta }$ which is independent of the index i of the different densities and lets us denote it by ${\mathit{\theta }}_g$. It is important to note that this assumption is satisfied if all the true densities $g_i$ belong to the model family so that $g_i=f_{i,\mathit{\theta }}$ for some common $\mathit{\theta }={\mathit{\theta }}_g$. The minimum Bregman divergence estimating equation is given by Equation (3), which is solved by the minimizer of $H_n\left(\mathit{\theta }\right)$ as defined in Equation (2). We now define, for $i=1,2,\dots$

$$H^{\left(i\right)}\left(\mathit{\theta }\right)=\int \left(f_{i,\mathit{\theta }}\left(x\right)B^{\prime }\left(f_{i,\mathit{\theta }}\left(x\right)\right)-B\left(f_{i,\mathit{\theta }}\left(x\right)\right)-B^{\prime }\left(f_{i,\mathit{\theta }}\left(x\right)\right)g_i\left(x\right)\right)dx,$$

(8)

so that for the best fitting parameter ${\mathit{\theta }}_g$, we have ${\nabla }_{\mathit{\theta }}{H}^{\left(i\right)}\left({\mathit{\theta }}_g\right)=\mathbf{0}$, for $i=1,2,\dots$ We also define, for each $i=1,2,\dots$, the $s\times s$ matrix $J^{\left(i\right)}$ with $(k,l)$-th entry given by

$$J_{kl}^{\left(i\right)}=E_{g_i}\left[{\nabla }_{kl}{V}_{i,\mathit{\theta }}\left(X_i\right)\right],$$

(9)

where $V_{i,\mathit{\theta }}\left(X_i\right)$ is as defined in Equation (2), ${\nabla }_{kl}{V}_{i,\mathit{\theta }}\left(X_i\right)={\displaystyle \frac{{\partial }^2{V}_{i,\mathit{\theta }}\left(X_i\right)}{\partial {\theta }_k\partial {\theta }_l}}$ is the $(k,l)$-th component of ${\nabla }_{\theta }{V}_{i,\theta }\left(X_i\right)$, $E_{g_i}[·]$ denotes expectation under the distribution specified by $g_i$, and ${\theta }_i$ denotes the i-th component of $\mathit{\theta }$. We also define

$${\Psi }_n={\displaystyle \frac{1}{n}}\sum _{i=1}^n{J}^{\left(i\right)},$$

(10)

and the matrix

$${\Omega }_n={\displaystyle \frac{1}{n}}\sum _{i=1}^n{V}_{g_i}\left[{\nabla }_{\mathit{\theta }}{V}_{i,\mathit{\theta }}\left(X_i\right)\right],$$

(11)

where $V_{g_i}$ denotes taking variance under the distribution specified by $g_i$. The matrix defined in Equation (9) has the following expression:

$$\begin{array}{cc}\hfill J^{\left(i\right)}=& \int u_{i,{\mathit{\theta }}_g}\left(x\right)u_{i,{\mathit{\theta }}_g}^T\left(x\right)w\left(f_{i,{\mathit{\theta }}_g}\left(x\right)\right)f_{i,{\mathit{\theta }}_g}\left(x\right)dx+\hfill \\ \hfill & \int \left({\nabla }_{\mathit{\theta }}{u}_{i,{\mathit{\theta }}_g}\left(x\right)+u_{i,{\mathit{\theta }}_g}\left(x\right)u_{i,{\mathit{\theta }}_g}^T\left(x\right)h_i\left(x\right)\right)\left(f_{i,{\mathit{\theta }}_g}\left(x\right)-g_i\left(x\right)\right)w\left(f_{i,{\mathit{\theta }}_g}\left(x\right)\right)dx,\hfill \end{array}$$

(12)

where $w\left(t\right)=B^{″}\left(t\right)\times t$, $w^{\prime }$ is the derivative of w w.r.t. its argument and

$$h_i\left(t\right)=\left(w^{\prime }\left(f_{i,\mathit{\theta }}\left(t\right)\right)f_{i,\mathit{\theta }}\left(t\right)\right)/w\left(f_{i,\mathit{\theta }}\left(t\right)\right).$$

Similarly, the matrix defined in Equation (11) has the expression

$$\begin{array}{cc}\hfill {\Omega }_n& ={\displaystyle \frac{1}{n}}\sum _{i=1}^n\int \left(u_{i,{\mathit{\theta }}_g}\left(x\right)u_{i,{\mathit{\theta }}_g}^T\left(x\right)w^2\left(f_{i,{\mathit{\theta }}_g}\left(x\right)\right)g_i\left(x\right)dx-{\xi }_i{\xi }_i^T\right),\hfill \end{array}$$

(13)

where ${\xi }_i=\int u_{i,{\mathit{\theta }}_g}\left(x\right)w\left(f_{i,{\mathit{\theta }}_g}\left(x\right)\right)g_i\left(x\right)dx.$ We will make the following assumptions to establish the asymptotic properties of the minimum Bregman divergence estimators. These are in the spirit of [6], with appropriate modifications to cover the general independent but non-homogeneous data case.

Assumption 1.

The support $\chi =\{x:f_{i,\mathit{\theta }}\left(x\right)>0\}$ is independent of i and θ for all i; the true densities $g_i$ are also supported on χ for all i.

Assumption 2.

There is an open subset ω of the parameter space Ω, containing the best fitting parameter ${\mathit{\theta }}_g$ such that for almost all $x\in \chi$, and all $\mathit{\theta }\in \Omega$, all $i=1,2,\dots$, the density $f_{i,\mathit{\theta }}\left(x\right)$ is thrice differentiable with respect to θ and the third partial derivatives are continuous with respect to θ.

Assumption 3.

For $i=1,\dots ,n$, both $\int [f_{i,\mathit{\theta }}\left(x\right)B^{\prime }\left(f_{i,\mathit{\theta }}\left(x\right)\right)-B\left(f_{i,\mathit{\theta }}\left(x\right)\right)]dx$ and $\int B^{\prime }\left(f_{i,\mathit{\theta }}\left(x\right)\right)g_i\left(x\right)dx$ can be differentiated thrice with respect to θ, and the derivatives can be taken under the integral sign.

Assumption 4.

For each $i=1,2,\dots$ the matrices $J^{\left(i\right)}$ are positive definite and ${\lambda }_0=\underset{n}{inf}[\mathit{min} \mathit{eigenvalue} \mathit{of} {\Psi }_n]$ is positive.

Assumption 5.

There exists a function $M_{jkl}^{\left(i\right)}\left(x\right)$ such that $\mid {\nabla }_{jkl}{V}_{i,\mathit{\theta }}\left(x\right)\mid \le M_{jkl}^{\left(i\right)}\left(x\right)\forall \mathit{\theta }\in \Omega \mathrm{and}\forall i=1,2,\dots$ where $n^{-1}{\sum }_{i=1}^n{E}_{g_i}\left[M_{jkl}^{\left(i\right)}\left(X_i\right)\right]=O\left(1\right)\forall j,k,l.$

Assumption 6.

For all $j,k$ we have

$$\begin{array}{cc}\hfill {\displaystyle \underset{N\to \infty }{lim}}{\displaystyle \underset{n}{sup}}& \{{\displaystyle \frac{1}{n}}{\displaystyle \sum _{i=1}^n}{E}_{g_i}(\mid {\nabla }_j{V}_{i,\mathit{\theta }}\left(X_i\right)\mid I(\mid {\nabla }_j{V}_{i,\mathit{\theta }}\left(X_i\right)\mid >N))\}=0.\hfill \\ \hfill {\displaystyle \underset{N\to \infty }{lim}}{\displaystyle \underset{n}{sup}}& \{\frac{1}{n}{\displaystyle \sum _{i=1}^n}{E}_{g_i}\left(\mid {\nabla }_{jk}{V}_{i,\mathit{\theta }}\left(X_i\right)-E_{g_i}\left({\nabla }_{jk}{V}_{i,\mathit{\theta }}\left(X_i\right)\right)\mid \right)\times \hfill \\ & I\left(\mid {\nabla }_{jk}{V}_{i,\mathit{\theta }}\left(X_i\right)-E_{g_i}\left({\nabla }_{jk}{V}_{i,\mathit{\theta }}\left(X_i\right)\right)\mid >N\right))\}=0.\hfill \end{array}$$

where $I\left(A\right)$ denotes the indicator variable associated with the event A.

Assumption 7.

For all $ϵ>0$, we have

$$\underset{n\to \infty }{lim}\left\{\sum _{i=1}^n{E}_{g_i}\left(\parallel {\Omega }_n^{-1/2}{\nabla }_{\mathit{\theta }}{V}_{i,\mathit{\theta }}\left(X_i\right){\parallel }^{2}I\left(\parallel {\Omega }_n^{-1/2}{\nabla }_{\mathit{\theta }}{V}_{i,\mathit{\theta }}\left(X_i\right)\parallel >ϵ\sqrt{n}\right)\right)\right\}=0.$$

Theorem 1.

Under Assumptions 1–7, the following results hold.

1. 

There exists a consistent sequence ${\widehat{\mathit{\theta }}}_n$ of roots satisfying the minimum Bregman divergence estimating equation given by Equation (3).

2. 

The asymptotic distribution of ${\Omega }_n^{-1/2}{\Psi }_n\left[\sqrt{n}({\widehat{\mathit{\theta }}}_n-{\mathit{\theta }}_g)\right]$ is s-dimensional normal with mean vector $\mathbf{0}$ and covariance $I_s$, the s-dimensional identity matrix.

The proof is provided in Appendix D.

Remark 3.

Setting $f_i=f\forall i$, we get the corresponding asymptotic properties of the minimum Bregman divergence estimator for i.i.d. case as given in Basu et al. [6]. In particular, choosing $B\left(x\right)=x^{1+\alpha }/\alpha$, we recover the asymptotic distribution of MDPDE(α). Assumptions 1–5 are similar to those given by Basu et al. [6], while Assumptions 6 and 7 are automatically satisfied by the dominated convergence theorem for i.i.d. data.

Remark 4.

Intuition regarding Assumptions 1–7: Conditions (1)–(4) are standard regularity assumptions required for the consistency and asymptotic normality of M-estimators, ensuring model identifiability, differentiability of the log-likelihood, and non-singularity of the Fisher Information Matrix. Conditions (5)–(7) are necessary to extend these results from the i.i.d. setting to the case of independent non-homogeneous observations. Specifically, (5) ensures the uniform boundedness of the third derivatives, allowing for valid Taylor expansions across varying densities $g_i$; (6) provides the uniform integrability required for the Law of Large Numbers to apply to the score and Hessian matrices; and (7) is the standard Lindeberg–Feller condition, which ensures that the sum of independent (but non-identical) score functions converges to a normal distribution by preventing any single observation from dominating the asymptotic variance. Ideally, the bounded nature of the EWD weight function ($w\in [0,1]$) facilitates the satisfaction of these moment conditions compared to unbounded influence functions. These conditions are widely satisfied by standard exponential families (e.g., normal, Poisson) under compact parameter spaces

3.3. Influence Function Analysis

Let $G_i$ ($g_i$) be the true distribution (density) for $Y_i,i=1,2,\dots ,n$ and $T_B(G_1,G_2,\dots ,G_n)$ be the minimum Bregman divergence functional obtained, under appropriate regularity conditions, as the solution of the system given by Equation (3). To derive the influence function of the minimum Bregman divergence estimator in the context of non-homogeneous data, we consider the set of contaminated distributions $G_{i,ϵ}=(1-ϵ)G_i+ϵ\Delta \left(t_i\right)$, where $ϵ\in (0,1)$ represents the contamination proportion and $\Delta \left(t_i\right)$ denotes the Dirac measure, representing the degenerate distribution at the point of contamination $t_i$ ($i=1,2,\dots ,n$). The associated contaminated density is given by $g_{i,ϵ}$. Let ${\mathit{\theta }}_0=T_B(G_1,G_2,\dots ,G_n)$ and let ${\mathit{\theta }}_{ϵ}^i=T_B(G_1,G_2,\dots ,G_{i-1},G_{i,ϵ},G_{i+1},\dots ,G_n)$ be the minimum Bregman divergence functional with contamination only in the i-th direction. Substituting ${\mathit{\theta }}_{ϵ}^i$ and $g_{i,ϵ}$ in the estimating Equation (3), differentiating with respect to $ϵ$ and evaluating the derivative at $ϵ=0$, we obtain the influence function of the functional which considers contamination only along the i-th direction to be

$$I{F}_i(t_i,T_B,G_1,G_2,\dots ,G_n)={\displaystyle \frac{{\Psi }_n^{-1}}{n}}\left[u_{i,\mathit{\theta }}\left(t_i\right)w\left(f_{i,\mathit{\theta }}\left(t_i\right)\right)-{\xi }_i\right],$$

where ${\Psi }_n$ is as defined in Equation (10), $w\left(t\right)=B^{″}\left(t\right)\times t$ and ${\xi }_i=\int u_{i,\mathit{\theta }}\left(x\right)w\left(f_{i,\mathit{\theta }}\left(x\right)\right)g_i\left(x\right)dx$. Letting ${\mathit{\theta }}_{ϵ}=T_B(G_{1,ϵ},G_{2,ϵ},\dots ,G_{n,ϵ})$ and proceeding similarly, we obtain the influence function with contamination at all the data points as

$$IF(t_1,\dots ,t_n,T_B,G_1,\dots ,G_n)={\displaystyle \frac{{\Psi }_n^{-1}}{n}}\sum _{i=1}^n\left[u_{i,\mathit{\theta }}\left(t_i\right)w\left(f_{i,\mathit{\theta }}\left(t_i\right)\right)-{\xi }_i\right].$$

(14)

In particular, letting $t_i=t$, $G_i=G$ and $f_i=f,\forall i$, we get back the influence function of the minimum Bregman divergence estimator for the i.i.d. case. In particular, for the MEWDE($\beta$), the influence function reduces to

$$IF(t,T_{\beta },G)=J^{-1}\left[u_{\mathit{\theta }}\left(t\right)\left[1-exp(f_{\mathit{\theta }}\left(t\right)/\beta )\right]-\xi \right],$$

(15)

where

$$\begin{array}{cc}\hfill J& =\int u_{\mathit{\theta }}\left(x\right)u_{\mathit{\theta }}^T\left(x\right)[1-exp(-f_{\mathit{\theta }}\left(x\right)/\beta )]f_{\mathit{\theta }}\left(x\right)dx,\hfill \\ \hfill \xi & =\int u_{\mathit{\theta }}\left(x\right)[1-exp(-f_{\mathit{\theta }}\left(x\right)/\beta )]f_{\mathit{\theta }}\left(x\right)dx.\hfill \end{array}$$

Figure 3 describes the theoretical influence function for $\widehat{\mu }\left(\beta \right)$—the MEWDE($\beta$) functional for the mean parameter of the contaminated normal distribution $(1-ϵ)N(\mu ,1)+ϵ{\Delta }_t$ for various contamination values t. For all $\beta >0$ considered here, we note their bounded and redescending nature.

4. Simulation Studies for MEWDE(β)

4.1. Introduction

  For i.i.d. data, when the true density g belongs to the model, i.e., $g=f_{\mathit{\theta }}$ for some $\mathit{\theta }\in \Omega$, let $\widehat{\mathit{\theta }}$ refer to the MEWDE of an unknown parameter $\mathit{\theta }$. For $\beta >0$, under the regularity conditions (1)–(7) outlined in Section 3.2, the score function satisfies the requirements of the Lindeberg–Feller Central Limit Theorem. Although this theorem [18] is typically invoked for non-identically distributed data, it remains applicable in our i.i.d. setting because the finite variance of the score function automatically satisfies the Lindeberg condition. Consequently, applying the standard Taylor expansion technique for M-estimators, we formally state that the asymptotic normality of $\sqrt{n}(\widehat{\mathit{\theta }}-\mathit{\theta })$ is an s- variate normal distribution with mean vector $\mathbf{0}$ and dispersion matrix given by $J^{-1}K{J}^{-1}$, where

$$\begin{array}{cc}\hfill J& =\int u_{\mathit{\theta }}\left(x\right)u_{\mathit{\theta }}^T\left(x\right)[1-exp(-f_{\mathit{\theta }}\left(x\right)/\beta )]f_{\mathit{\theta }}\left(x\right)dx,\hfill \\ \hfill K& =\int u_{\mathit{\theta }}\left(x\right)u_{\mathit{\theta }}^T\left(x\right){[1-exp(-f_{\mathit{\theta }}\left(x\right)/\beta )]}^2{f}_{\mathit{\theta }}\left(x\right)dx-\xi {\xi }^T,\hfill \\ \hfill \xi & =\int u_{\mathit{\theta }}\left(x\right)[1-exp(-f_{\mathit{\theta }}\left(x\right)/\beta )]f_{\mathit{\theta }}\left(x\right)dx.\hfill \end{array}$$

(16)

As $\beta \to 0$, both J and K reduce to the Fisher information matrix. We now consider different parametric families and compare the performance of MEWDEs and MDPDEs under different contamination scenarios.

4.2. Tuning Parameter Selection

In minimum EWD estimation, small values of $\beta$ provide greater model efficiency, while large values of $\beta$ provide greater outlier stability and protection against small model violations. Given any real dataset, we must choose the “optimal”, data-based tuning parameter $\beta$ so that the procedure has the right amount of balance for the dataset in question. We follow the approach of [19] to derive the optimal estimate of the tuning parameter. This approach constructs an empirical estimate of the Mean Squared Error as a function of the tuning parameter $\beta$ and a pilot estimator ${\mathit{\theta }}^P$ given by

$$\begin{array}{c}\hfill \widehat{{MSE}_{\beta }}\left({\mathit{\theta }}^P\right)={\left({\widehat{\mathit{\theta }}}_{\beta }-{\mathit{\theta }}^P\right)}^T\left({\widehat{\mathit{\theta }}}_{\beta }-{\mathit{\theta }}^P\right)+{\displaystyle \frac{1}{n}}\mathrm{tr}\left({\Psi }_n^{-1}\left({\widehat{\mathit{\theta }}}_{\beta }\right){\Omega }_n\left({\widehat{\mathit{\theta }}}_{\beta }\right){\Psi }_n^{-1}\left({\widehat{\mathit{\theta }}}_{\beta }\right)\right),\end{array}$$

(17)

where $\mathrm{tr}(·)$ is the trace of a matrix and ${\Psi }_n$ and ${\Omega }_n$ are the matrices defined in Equations (10) and (11), respectively, evaluated at $\mathit{\theta }={\widehat{\mathit{\theta }}}_{\beta }=\mathrm{MEWDE}\left(\beta \right)$. Further, tr(·) denotes the trace of a matrix. By minimizing the objective function given in Equation (17) over $\beta >0$, we get a data driven “optimal” estimate of the tuning parameter. Ref. [19] proposes the minimum $L_2$ estimator as the pilot estimator in the above calculation.

4.3. Simulation Scheme

To rigorously evaluate the finite-sample performance of the proposed MEWDE, we compared it against the minimum density power divergence estimator (MDPDE) and the classical Maximum Likelihood Estimator (MLE). We adopted an “Oracle” tuning approach for the simulation study. For each estimator and each simulation scenario (combination of sample size n and contamination level $ϵ$), we performed a grid search to identify the optimal tuning parameter (${\alpha }_{OPT}$ for DPD, ${\beta }_{OPT}$ for EWD) that minimizes the empirical Mean Squared Error (MSE). This approach reports the best possible performance achievable by each estimator, isolating the theoretical capability of the divergence measure from the variability of data-driven tuning selection. Note that the practical data-driven algorithm for selecting $\beta$ in real applications is detailed in Section 4.2.

4.4. Results

We first define the empirical finite sample relative efficiency (FSRE) of the MEWDE (or MDPDE) as the ratio of MSE(MLE) to MSE(MEWDE) (or to MSE(MDPDE)). Under this metric, a value of $1.0$ indicates efficiency equivalent to the MLE, while values $<1.0$ indicate superior performance (error reduction) in the presence of contamination. To provide a comprehensive benchmark, we also report the performance of classical robust methods, specifically Huber’s M-estimator and Tukey’s Bisquare.

We consider three separate simulation designs involving (a) estimation of the mean of a univariate normal distribution with known standard deviation, (b) estimation of the standard deviation of a univariate normal distribution with known mean, and (c) estimation of the mean parameter of an exponential distribution. For simulation (a), the true distribution is taken to be $N(0,1)$ and the contaminating distribution is $N({\mu }_c,1)$. We run simulations for ${\mu }_c=3\mathrm{and}5$ and estimate the mean under the $N(\mu ,1)$ model. Our findings are presented in

Table 1. Finite sample relative efficiency (FSRE) of the MEWDE compared to the MLE and MDPDE under varying contamination levels ($ϵ$) and sample sizes (n).

$\mathit{ϵ}$	n	Relative Efficiency
		Huber	Bisquare	MDPDE (Oracle)	MEWDE (Oracle)
0.0	50	1.021	1.045	1.002	1.012
	100	1.002	1.013	1.006	1.004
	200	1.035	1.055	1.044	1.003
0.1	50	0.315	0.123	0.087	0.078
	100	0.263	0.065	0.047	0.039
	200	0.231	0.040	0.030	0.024
0.2	50	0.678	0.404	0.261	0.027
	100	0.582	0.265	0.023	0.016
	200	0.555	0.200	0.014	0.008
0.3	50	0.920	0.792	0.395	6.413
	100	0.918	0.788	0.093	0.995
	200	0.923	0.804	0.014	0.005

. Table 1 presents the finite sample relative efficiency (FSRE) results, calculated as the ratio of the MSE of the MLE to the MSE of the robust estimator. Consequently, a value greater than 1 indicates the robust method is less efficient than the MLE, while values less than 1 (common under contamination) indicate superior performance. The results cover the pure location model with sample sizes $n=50,100,200$ and contamination proportions $ϵ=0,0.10,0.20,0.30$.

Table 1 demonstrates that under an independent “Oracle” tuning protocol, the MEWDE offers a superior efficiency–robustness trade-off in the most common contamination regimes. In the absence of outliers ($ϵ=0$), the estimator retains high efficiency comparable to the MLE (FSRE $\approx 1.0$), confirming that robustness does not come at the cost of performance under the null model. Crucially, under moderate contamination ($ϵ\in [0.1,0.2]$), the MEWDE consistently outperforms both the MDPDE and classical robust methods; for example, at $n=50$ and $ϵ=0.2$, the MEWDE achieves a ten-fold reduction in MSE compared to the MDPDE (0.027 vs. 0.261), while the MDPDE exhibits greater stability in small-sample, severe-contamination settings ($n=50,ϵ=0.3$); the MEWDE recovers its superiority as sample size increases ($n=200$), suggesting that the exponential weight function provides sharper discrimination against outliers when sufficient data is available to stabilize the density estimate. For completeness, the full tabulated results and efficiency comparisons for Scenarios (b) and (c) are provided in Appendix C. These additional results exhibit trends consistent with Scenario (a), confirming the method’s stability across different distributional assumptions.

4.5. Sensitivity to Tuning Parameter Misspecification

A practical concern for robust M-estimation is the sensitivity of the estimator to the choice of the tuning parameter $\beta$, while our primary results rely on an Oracle selection, practitioners require guidance on a “safe” range for $\beta$ that yields robust performance without requiring prior knowledge of the contamination level. To address this, we examined the efficiency ratio defined as ${\mathrm{MSE}}_{\mathrm{MLE}}/{\mathrm{MSE}}_{\mathrm{MEWDE}}$ across a grid of tuning parameters $\beta \in [0.1,1.0]$. In this metric, values greater than $1.0$ indicate that the proposed estimator outperforms the non-robust MLE. We varied the sample size $n\in \{50,100,200\}$ and contamination proportion $ϵ\in \{0.0,0.1,0.2,0.3\}$ (using a mean-shift contamination $N(3,1)$) to identify the regions of stability and breakdown. The sensitivity of the estimator is illustrated in Figure 4. In each panel, the black curve represents the MSE profile as $\beta$ varies, while the vertical red dashed line marks the Oracle optimal $\beta$. The results (summarized in Figure 4) reveal three distinct performance regimes that offer concrete guidance for parameter selection:

• Efficiency Cost (Clean Data, $ϵ=0$): In the absence of outliers, the efficiency ratio is consistently below $1.0$, reflecting the expected cost of robustness. However, for small tuning parameters ($\beta \in [0.1,0.3]$), the ratio remains high (approx. $0.75-0.98$ for $n\ge 50$), confirming that the MEWDE retains the majority of the statistical information when the model is correctly specified.
• Robustness Regime ($ϵ\in [0.1,0.2]$): Under moderate contamination, the estimator demonstrates a wide “basin of stability.” The MEWDE consistently dominates the MLE, achieving efficiency ratios between $3.0$ and $17.0$ across the entire range of tested $\beta$. This indicates that for the most common contamination scenarios, the method is highly insensitive to parameter misspecification; any $\beta \in [0.1,1.0]$ yields superior results.
• Severe Contamination Limit ($ϵ=0.3$): A critical divergence occurs under severe contamination. For small sample sizes ($n=50$), the estimator breaks down regardless of $\beta$, as the minority inliers are insufficient to anchor the density. However, for moderate sample sizes ($n=100$), we observe a clear transition: small tuning parameters ($\beta \in [0.1,0.4]$) maintain robustness (ratio $>7.0$), while larger values ($\beta \ge 0.5$) lead to performance degradation.

Figure 4. Sensitivity analysis showing the Mean Squared Error (MSE) of the MEWDE as a function of the tuning parameter $\beta$ for different sample sizes.

Figure 4. Sensitivity analysis showing the Mean Squared Error (MSE) of the MEWDE as a function of the tuning parameter $\beta$ for different sample sizes.

Based on these findings, we recommend $\beta \in [0.1,0.3]$ as a default operating range. This interval minimizes efficiency loss on clean data while providing maximum protection against severe contamination in intermediate sample sizes ($n=100$), avoiding the breakdown risks associated with larger smoothing parameters.

4.6. Performance of Data-Driven Tuning

A common critique of minimum divergence methods is the reliance on a tuning parameter $\beta$ that must be selected by the user, while the primary simulation results presented in Table 1 utilized an “Oracle” approach (selecting the $\beta$ that minimizes the actual squared error) to establish the theoretical limit of the method, this is impossible in practical applications where the true parameter is unknown.

To assess the practical feasibility of the MEWDE, we implemented the data-driven tuning algorithm proposed by Warwick and Jones [19]. This method selects the optimal tuning parameter $\widehat{\beta }$ by minimizing an empirical estimate of the Asymptotic Mean Squared Error (AMSE):

$$\widehat{\beta }=arg\underset{\beta }{min}\left[{({\widehat{\mathit{\theta }}}_{\beta }-{\widehat{\mathit{\theta }}}_{pilot})}^2+{\displaystyle \frac{\widehat{Var}\left({\widehat{\mathit{\theta }}}_{\beta }\right)}{n}}\right],$$

(18)

where ${\widehat{\mathit{\theta }}}_{\beta }$ is the MEWDE computed with tuning parameter $\beta$, and $\widehat{Var}\left({\widehat{\mathit{\theta }}}_{\beta }\right)$ is the sandwich variance estimate. Crucially, this criterion requires a pilot estimator ${\widehat{\mathit{\theta }}}_{pilot}$ to approximate the bias. For this simulation, we following [19]’s proposal to use the minimum $L_2$ estimator as ${\widehat{\mathit{\theta }}}_{pilot}$. The simulation setting mirrors the pure location model described in Section 4.3. We generated independent observations from a contaminated normal mixture model:

$$g\left(x\right)=(1-ϵ)\varphi \left(x\right)+ϵ\varphi (x-5),$$

where $\varphi (·)$ denotes the standard normal density. We evaluated the performance across sample sizes $n\in \{25,100,500\}$ and contamination proportions $ϵ\in \{0.00,0.05,0.10,0.20\}$. For each replicate, the data-driven algorithm selected $\widehat{\beta }$ from the grid $\{0.1,0.2,\dots ,1.0\}$ without knowledge of the true parameter or contamination level. The results of this comparison are presented in

Table 2. Comparison of Mean Squared Error (MSE) between the Oracle-tuned MEWDE and the data-driven MEWDE (using the Warwick–Jones algorithm with minimum $L_2$ pilot).

$\mathit{ϵ}$	n	MSE (Oracle)	MSE (Data-Driven)	% Loss
0.00	25	0.0492	0.0612	24.41%
	100	0.0125	0.0155	24.06%
	500	0.0023	0.0027	19.31%
0.05	25	1.1244	1.1893	5.78%
	100	0.5593	0.5622	0.53%
	500	0.2582	0.2591	0.34%
0.10	25	1.3599	1.3746	1.08%
	100	2.0794	2.0843	0.23%
	500	1.7192	1.7195	0.02%
0.20	25	1.7120	1.7791	3.92%
	100	4.2975	4.2998	0.05%
	500	9.4143	9.4145	0.00%

.

The comparison between the Oracle and data-driven MEWDE is presented in Table 2. In the pure data scenario ($ϵ=0$), the data-driven approach incurs an efficiency loss of approximately 19–24% compared to the Oracle. However, as contamination is introduced, the performance of the data-driven tuning rapidly converges to that of the Oracle. For $ϵ\ge 0.10$, the percentage loss drops significantly, often falling below 1% (see Table 2). In these cases, the presence of outliers forces both the Oracle and the data-driven selector to adopt robust tuning parameters, eliminating the gap between the theoretical optimum and the practical estimate. This confirms that the Warwick–Jones algorithm, anchored by the minimum $L_2$ estimator, effectively adapts to the data structure, providing necessary robustness without requiring prior knowledge of the contamination level.

4.7. Stability in Multivariate Settings

To address concerns regarding the performance of the MEWDE as the dimensionality of the parameter space increases, we extended our simulation study to a multivariate linear regression setting. We considered models with $p\in \{2,5,10\}$ predictors and contamination rates of $ϵ\in \{0,0.1,0.2\}$. As shown in

Table 3. Parameter estimation error (sum of squared errors, $\parallel \widehat{\beta }{-\beta \parallel }_2^2$) for multivariate linear regression with p predictors. Data were generated with 10% and 20% vertical outliers from $N(20,1)$.

$\mathit{ϵ}$	n	p	MLE	MEWDE
				$\mathit{\beta }=\mathbf{0}.\mathbf{1}$	$\mathit{\beta }=\mathbf{0}.\mathbf{5}$
0.0	50	2	0.045	0.057	0.069
	100	5	0.054	0.071	0.091
	50	10	0.253	0.413	0.606
	100	10	0.111	0.156	0.209
	200	10	0.053	0.067	0.082
0.1	50	2	1.675	0.057	0.066
	100	5	2.287	0.068	0.079
	50	10	10.435	0.418	0.592
	200	10	2.168	0.070	0.082
0.2	50	2	3.418	0.056	0.063
	100	5	4.218	0.075	0.084
	50	10	20.555	0.406	0.518
	200	10	4.400	0.072	0.080

, the MLE suffers from severe degradation as dimensionality and contamination increase; for example, with $n=50$, $p=10$ and $20\%$ contamination, the MLE squared error explodes to $20.55$. In contrast, the MEWDE ($\beta =0.1$) maintains remarkable stability with an error of $0.406$, comparable to its performance on clean data. Furthermore, the results demonstrate that $\beta =0.1$ provides an excellent trade-off, achieving near-optimal efficiency on clean data while offering full protection against vertical outliers.

4.8. Computational Complexity

The computational cost of the MEWDE is dominated by the evaluation of the objective function and its gradient at each step of the iterative solver. For a sample size n, the evaluation of the estimating equation requires summing the score function over all observations; this operation is $\mathcal{O}\left(n\right)$. The integral term required for Fisher consistency depends only on the parameters, not the sample size, and thus adds a constant overhead $\mathcal{O}\left(1\right)$ per iteration (assuming fixed dimensionality). Consequently, if the numerical solver (e.g., Newton–Raphson) requires k iterations to converge, the total complexity for a fixed tuning parameter is $\mathcal{O}\left(kn\right)$. When selecting the optimal tuning parameter ${\beta }_{OPT}$ via the data-driven grid search described above, the estimator is re-computed for each of the G candidate values in the grid. The total complexity for the tuning procedure is therefore $\mathcal{O}\left(Gkn\right)$. Since the grid size G and the number of iterations k are typically small and fixed relative to the sample size, the overall procedure remains linear in n, i.e., $\mathcal{O}\left(n\right)$. This ensures that the data-driven selection of $\beta$ remains computationally feasible even for large datasets, comparable to standard likelihood-based methods.

Remark 5.

The estimators were computed by solving the system of estimating equations defined in Equation (3). We utilized the multiroot  function from the  rootSolve  package in R [20], which implements a Newton–Raphson method with diagonal Jacobian approximation. The convergence tolerance was set to ${10}^{-8}$. This root-finding approach proved to be numerically stable and computationally efficient, with average runtimes comparable to standard M-estimation routines and no significant overhead compared to the MDPDE.

Remark 6.

Under light or no contamination, the MEWDE and MDPDE exhibit nearly indistinguishable performance, as expected given their shared first-order efficiency at the model. The practical advantage of the EWD becomes apparent primarily in moderate to heavy contamination settings, where bounded weighting prevents both outlier influence and inlier over-emphasis.

5. Modeling Real Data

To ensure a comprehensive evaluation, we compare our proposed MEWDE against the standard MDPDE as well as classical robust estimators, specifically Huber and Tukey’s Bisquare estimators. We also report the ML + D (Outlier-Deleted Maximum Likelihood Estimator), which serves as a “clean” benchmark calculated after manually removing the identified outliers from the dataset.

5.1. Shoshoni Rectangles

We consider the data on Shoshoni rectangles presented and analyzed by [21]. The parameter estimates for the Shoshoni dataset are summarized in

Table 4. Parameter estimates for the Shoshoni dataset using classical and robust estimators. $\mathcal{N}(\mu ,{\sigma }^2)$ distribution is fitted using MLE, Huber, Bisquare, MDPDE ($\alpha =0.5$), and MEWDE ($\alpha =0.4$).

Method	$\widehat{\mathit{\mu }}$	$\widehat{\mathit{\sigma }}$
MLE	0.6605	0.0902
Huber	0.6421	0.0519
Bisquare	0.6345	0.0519
MDPDE ($\alpha =0.5$)	0.6314	0.0472
MEWDE ($\alpha =0.4$)	0.6060	0.0421

and visualized in Figure 5. The results clearly illustrate the sensitivity of the classical Maximum Likelihood Estimator (MLE) to extreme values compared to the robust alternatives. The MLE yields the largest location and scale estimates (${\widehat{\mu }}_{\mathrm{MLE}}\approx 0.661$, ${\widehat{\sigma }}_{\mathrm{MLE}}\approx 0.090$), indicating that the fit is being pulled toward the right-skewed outliers inherent in the data. This inflation of the standard deviation results in a density curve that is overly dispersed and fails to capture the mode of the bulk data. In contrast, the M-estimators (Huber and Bisquare) and the divergence-based estimators (MDPDE and MEWDE) provide significantly tighter fits. Both the Huber and Bisquare methods reduce the location estimate to approximately $0.63-0.64$ and nearly halve the estimated scale to $\widehat{\sigma }\approx 0.052$. The divergence-based estimators demonstrate the strongest robustness profiles. The MDPDE ($\alpha =0.5$) and MEWDE ($\beta =0.4$) produce the smallest scale estimates ($\widehat{\sigma }\approx 0.047$), suggesting they have successfully trimmed the influence of the outliers to focus on the central density. Notably, the MEWDE provides the most distinct location estimate (${\widehat{\mu }}_{\mathrm{MEWDE}}\approx 0.606$), shifting the center further to the left than the other methods. This suggests that the MEWDE with $\beta =0.4$ is highly effective at identifying the core underlying distribution, virtually ignoring the influence of the larger values that skew the standard MLE.

5.2. Drosophila Offspring Counts

We compare the performance of the MEWDE and MDPDE in the context of data on fruit flies (see [22]). In this experiment male flies were sprayed with a chemical, and then made to mate with unexposed females. The response, for each father fly, was the number of daughter flies having a recessive lethal mutation in the X-chromosome.

The frequencies of these responses (presented in the first row of

Table 5. Fitted frequencies for Drosophila data using MLE, MDPDE (denoted by D($\alpha$)) and MEWDE (denoted by E($\beta$)), where $\alpha$ and $\beta$ are tuning parameters for MDPDE and MEWDE.

Count	0	1	2	3	4	≥5	$\widehat{\mathit{\lambda }}$
Observed	23	7	3	0	0	1 (91)	–
MLE.	$1.596$	$4.882$	$7.467$	$7.613$	$5.822$	$6.620$	$3.059$
D($0.10$)	$22.981$	$9.002$	$1.763$	$0.230$	$0.023$	$0.002$	$0.392$
D($0.50$)	$23.375$	$8.759$	$1.641$	$0.205$	$0.019$	$0.002$	$0.375$
D($0.75$)	$23.549$	$8.649$	$1.588$	$0.194$	$0.018$	$0.001$	$0.367$
E($0.001$)	$22.894$	$9.055$	$1.791$	$0.236$	$0.023$	$0.002$	$0.396$
E($0.02$)	$22.614$	$9.222$	$1.880$	$0.256$	$0.026$	$0.002$	$0.408$
E($0.25$)	$23.712$	$8.545$	$1.540$	$0.185$	$0.017$	$0.001$	$0.360$
$L_2$	$23.609$	$8.611$	$1.570$	$0.191$	$0.017$	$0.001$	$0.365$
MLE + D	$22.93$	$9.03$	$1.78$	$0.23$	$0.02$	0	$0.39$

) are modeled as Poisson variables, and the estimates of the Poisson mean parameter $\lambda$ (as well as the estimated frequencies), using several members of the DPD and EWD families, the MLE and the MLE + D (outlier-deleted MLE) are presented in Table 5. The single extreme value at 91 is treated as the obvious outlier. Both sets of estimators have comparable (satisfactory) performance, which adequately discount the outlier. To visually assess the impact of the extreme outlier on model adequacy, Figure 6 overlays the fitted Poisson densities on the observed count frequencies, illustrating the divergence-based estimators’ superior fit to the bulk data compared to the MLE.

5.3. Homicide from Firearm Use and GDP

We consider modeling age-standardized national firearm-related homicide rates in 23 Western countries as a function of per-capita gross domestic product as of 2017. Country-specific information on firearm-related homicide rates (from [23]) and GDP (from [24]) were obtained. Figure 7 displays the scatter plot of firearm-related homicide rates against per-capita GDP, revealing the United States as a significant outlier that severely distorts the standard MLE, resulting in a counter-intuitive positive slope estimate of $1.45\times {10}^{-5}$. In contrast, as summarized in

Table 6. Regression coefficient estimates and scale estimates for the homicide rate dataset. The response variable is the homicide rate, and the predictor is GDP.

Parameter	MLE	ML + D	Huber	Bisquare	MEWDE (0.3)	MEWDE (0.5)	MDPDE (0.3)	MDPDE (0.5)
Intercept (${\beta }_0$)	−0.293	0.356	0.333	0.357	0.348	0.351	0.360	0.364
Slope (${\beta }_1\times {10}^5$)	1.45	−0.30	−0.25	−0.31	−0.31	−0.32	−0.32	−0.33
Sigma ($\sigma$)	0.959	0.116	0.115	0.112	0.102	0.103	0.109	0.108

, all robust estimators—including Huber, Bisquare, MDPDE, and the proposed MEWDE—successfully identify and downweight this leverage point, recovering the negative relationship observed in the outlier-deleted benchmark (ML + D). Notably, MEWDE achieves the lowest scale estimate ($\sigma \approx 0.102$) among all robust competitors, indicating it provides the tightest fit to the majority of the data.

5.4. Solubility of Alcohols in Water

We consider fitting a multiple linear regression model to the dataset concerning alcohol solubility in water (see [25]). The dataset gives, for 44 aliphatic alcohols, the logarithm of their solubility together with three physicochemical characteristics (namely, solvent accessible surface-bounded molecular volume (SAG), mass and volume). The interest is in predicting the solubility. Following the authors’ suggestion of fitting an MM regression-based model to the data, we observe that four data points (roughly $10\%$ of the dataset) are assigned much smaller ‘robustness weights’ as compared to the remaining 40 data points. Treating these four observations as outliers, we obtain the Huber and Bisquare estimates of the regression coefficients and error standard deviation. We also compute the robust LMS estimate. In order to estimate the error s.d. $\sigma$, we compute $\widehat{\sigma }=\mathrm{median}|r_i-\mathrm{median}\left(r_i\right)|/0.67449$. Finally, we compute minimum EWD($\beta$) regression parameter estimates for various values of $\beta$. Our findings are presented in

Table 7. Estimated regression parameters for alcohol solubility data.

Estimates	MLE	LMS	E(0.1)	E(0.4)	E(0.7)	Huber	Bisquare
Intercept	$8.777$	$3.617$	$5.883$	$3.974$	$5.444$	7.990	7.919
SAG	$0.014$	$0.177$	$0.110$	$0.163$	$0.129$	$0.038$	$0.041$
Volume	$-0.040$	$-0.191$	$-0.133$	$-0.179$	$-0.152$	$-0.063$	$-0.066$
Mass	$0.027$	$0.248$	$0.172$	$0.235$	$0.206$	$0.063$	$0.068$
Error s.d.	$0.504$	$0.405$	$0.372$	$0.145$	$0.221$	$0.541$	$0.552$

.

As a visual inspection is not possible for the fits in this multiple linear regression model, the coefficients of Table 7 are not sufficient on their own to give a full idea about how stable and outlier-resistant the fits are. By means of Figure A5 and Figure A6 in the Appendix B, we examine the residuals of each of the fits for the data and explore how well they fare in terms of separating out the outliers. Figure A5 and Figure A6 demonstrate a particular MEWDE and the LMS estimate are much more successful in making the outliers stand out and giving stable fits compared to the least squares fit provided by the MLE.

5.5. Tuning Parameters for the Data Examples Considered

Through the procedure outlined in Section 4.2, the optimal tuning parameter for the data on Shoshoni rectangles is found to be ${\beta }_{\mathrm{OPT}}=0.43$, and the associated estimated parameters are $\widehat{\mu }=0.63$ and $\widehat{\sigma }=0.05$. The corresponding (sample size-scaled) asymptotic Mean Squared Error is $5.07\times {10}^{-3}$. For the data on Drosophila fruit flies, the optimal tuning parameter is found to be ${\beta }_{OPT}=0.08$, and the estimated mean parameter is $\widehat{\lambda }=0.377$. The corresponding (sample size-scaled) asymptotic Mean Squared Error is $0.46$. For the data on firearm-related homicide and GDP, the optimal tuning parameter is ${\beta }_{OPT}=1.6$, and the corresponding estimated regression parameters (intercept, GDP, error standard deviation) are $(0.416,-3.932\times {10}^{-6},0.066)$. Finally, for the data on alcohol solubility, the optimal tuning parameter is ${\beta }_{OPT}=0.66$, and the corresponding estimated regression parameters intercept, SAG, volume, mass, error standard deviation are $(6.084,0.112,-0.135,0.174,0.062)$, respectively.

6. Testing of Hypotheses

6.1. Introduction

Here we develop general robust tests of the hypotheses based on Bregman divergences. This generalizes the works of [15,16]. We establish the asymptotic null distribution of the proposed test statistic and apply the theory developed to a real-life dataset. Our focus will remain on EWD($\beta$).

Unlike the previous sections, we will consider the case of i.i.d. data only. We begin with an identifiable parametric family of probability measures $P_{\mathit{\theta }}$ on a measurable space $\{\chi ,\mathcal{A}\}$ with an open parameter space $\Omega \subseteq {\mathbb{R}}^s,s\ge 1.$ Measures $P_{\mathit{\theta }}$ are described by densities $f_{\mathit{\theta }}=d{P}_{\mathit{\theta }}/d\mu$, absolutely continuous with respect to a dominating $\sigma$-finite measure $\mu$ on $\chi$. We have an i.i.d. sample of size n given by $X_1,X_2,\dots ,X_n$ from a density belonging to the family ${\mathcal{F}}_{\mathit{\theta }}=\{f_{\mathit{\theta }}:\mathit{\theta }\in \Omega \}$. We will assume that the support of the distribution is independent of $\mathit{\theta }$. The hypothesis of interest is $H_0:\mathit{\theta }\in {\Omega }_0\mathrm{against}{H}_1:\mathit{\theta }\notin {\Omega }_0.$ We use the common approach where the restricted parameter space specified by $H_0$ can be rewritten by a set of $r<s$ restrictions of the form

$$\mathit{m}\left(\mathit{\theta }\right)={\mathbf{0}}_r$$

(19)

on $\Omega$, where $\mathit{m}:{\mathbb{R}}^s\to {\mathbb{R}}^r$ is a vector valued function such that the $s\times r$ matrix $M\left(\mathit{\theta }\right)={\displaystyle \frac{\partial {\mathbf{m}}^{\mathbf{T}}\left(\mathit{\theta }\right)}{\partial \mathit{\theta }}}$ exists and is continuous in $\mathit{\theta }$ and rank($M\left(\mathit{\theta }\right)$) $=r$.

6.2. The Test Statistic

To perform this test of the hypotheses, we first obtain ${\widehat{\mathit{\theta }}}_{B_1}$, the unrestricted minimum Bregman divergence estimator for a given $B_1$ function and then obtain the restricted estimator ${\tilde{\mathit{\theta }}}_{B_1}$, subject to the constraints of Equation (19). We then examine the family of Bregman divergence test statistics (BDTS)

$$T_{B_2}({\widehat{\mathit{\theta }}}_{B_1},{\tilde{\mathit{\theta }}}_{B_1})=2n\times D_{B_2}(f_{{\widehat{\mathit{\theta }}}_{B_1}},f_{{\tilde{\mathit{\theta }}}_{B_1}}),$$

(20)

where $D_{B_2}(g,f)$ is the Bregman divergence between two densities g and f, defined in Equation (1) with $B_2$ as the B function. We will consider the functions $B_1$ and $B_2$ to have the same functional form (e.g., as given by the exponentially weighted divergences) only differing, if at all, in the values of their tuning parameters. Thus, in the derivation of the asymptotic distribution of the test statistics, the functions $B_1$ and $B_2$ are allowed to be different. In practice, however, a single, suitably chosen common function B will generally work well in most cases.

In addition to Assumptions 1–5 presented in Section 3.2, we make the following assumption.

Assumption 8.

For all $\mathit{\theta }\in \omega$, the partial derivatives ${\partial }^2{m}_l\left(\mathit{\theta }\right)/\partial {\mathit{\theta }}_j\partial {\mathit{\theta }}_k$ are bounded for all j, k and l, where $m_l(·)$ is the l-th element of $\mathit{m}(·)$.

Theorem 2.

Under Assumptions 1–5 and 8, and assuming that the true distribution belongs to the model, i.e., $G=F_{{\mathit{\theta }}_g}$ for some ${\mathit{\theta }}_g\in \Omega$ which satisfies the set of constraints given by Equation (19), the constrained minimum Bregman divergence estimator ${\tilde{\mathit{\theta }}}_{n,B_1}$ has the following properties: the underlying B function is denoted by $B_1$.

1. 

Consistency: ${\tilde{\mathit{\theta }}}_{n,B_1}\stackrel{P}{\to }{\mathit{\theta }}_{\mathbf{g}}$ as $n\to \infty$.

2. 

Asymptotic normality: The asymptotic null distribution of $\sqrt{n}({\tilde{\mathit{\theta }}}_{n,B_1}-{\mathit{\theta }}_g)$ is s-dimensional multivariate normal with the zero mean vector and an $s\times s$ dispersion matrix ${\Sigma }_{B_1}=P_{B_1}{K}_{B_1}{P}_{B_1}$, where $K_{B_1}$ is the i.i.d. analogue of the ${\Omega }_n$ matrix defined by Equation (11) with $B_1$ serving as the B function. The $P_{B_1}$ matrix is defined as $P_{B_1}=J_{B_1}^{-1}-Q_{B_1}M{J}_{B_1}^{-1}$, where $J_{B_1}$ is the i.i.d. analogue of the ${\Psi }_n$ matrix defined by Equation (10). Further, $Q=J_{B_1}^{-1}M{\left[M^T{J}_{B_1}^{-1}M\right]}^{-1}$; $M=M\left(\mathit{\theta }\right)$ is as defined in Section 6.1.

The proof is provided in Appendix D.

Remark 7.

Theorem 2 extends Theorem 1 in the context of i.i.d. data. Under the setup of Theorem 1, M becomes a null matrix and consequently, $P_{B_1}=J_{B_1}^{-1}$ and the asymptotic dispersion matrix of the unrestricted minimum Bregman divergence estimator assumes the form specified by Theorem 1.

Theorem 3 presents the asymptotic distribution of the test statistic defined in Equation (20).

Theorem 3.

Under Assumptions 1–5 and 8, the asymptotic distribution of the test statistic defined in Equation (20) is identical with, under the null hypothesis specified in Equation (19), the distribution of the random variable

$$\sum _{i=1}^k{\lambda }_i(B_1,B_2,\mathit{\theta })Z_i^2,$$

(21)

where $Z_1,\dots ,Z_k$ are independent standard normal variables and ${\lambda }_i(B_1,B_2,\mathit{\theta })$ for $i=1,\dots ,k$ are the nonzero eigenvalues of the matrix $A_{B_2}{B}_{B_1}{K}_{B_1}{B}_{B_1},$ and k is the rank of the matrix $B_{B_1}{K}_{B_1}{B}_{B_1}{A}_{B_2}{B}_{B_1}{K}_{B_1}{B}_{B_1}.$ The $(i,j)$-th element of $A_{B_2}$ is defined as follows:

$$A_{B_2}(i,j)=\int B_2^{″}\left(f_{{\mathit{\theta }}_{\mathbf{g}}}\left(x\right)\right)\left[{\displaystyle \frac{\partial f_{{\mathit{\theta }}_{\mathbf{g}}}\left(x\right)}{\partial {\mathit{\theta }}_i}}{\displaystyle \frac{\partial f_{{\mathit{\theta }}_{\mathbf{g}}}\left(x\right)}{\partial {\mathit{\theta }}_j}}\right]dx$$

and the matrix $B_{B_1}$ is equal to

$$J_{B_1}^{-1}M{\left[M^T{J}_{B_1}^{-1}M\right]}^{-1}{M}^T{J}_{B_1}^{-1}.$$

The proof is provided in Appendix D.

Remark 8.

We note that the ranks of $B_{B_1}{K}_{B_1}{B}_{B_1}{A}_{B_2}{B}_{B_1}{K}_{B_1}{B}_{B_1}$, $B_{B_1}{K}_{B_1}{B}_{B_1}$ and M are simultaneously equal to r.

Remark 9.

An easy way to approximate the required critical region of the above test is outlined here. From Theorem 3 it is obvious that the k eigenvalues described are functions of ${\mathit{\theta }}_g$. Under the null, they can be consistently estimated by plugging in ${\tilde{\mathit{\theta }}}_{B_1}$ in place of ${\mathit{\theta }}_g$. Let these estimated eigenvalues be ${\widehat{\lambda }}_1,\dots ,{\widehat{\lambda }}_k$. Generating k independent observations $Z_1,\cdots ,Z_k$ from the $N(0,1)$ distribution, one can approximate the quantiles of ${\sum }_i{\widehat{\lambda }}_i{Z}_i^2$ by replicating this procedure a large number of times, which can then serve as consistent estimates of the quantiles of the limiting variable in Equation (21).

Remark 10.

An approximate form of the power function of the test statistic can be obtained by following the steps outlined by Theorem 7 of [16].

Remark 11.

While Theorem 3 derives the asymptotic null distribution of the Bregman Divergence Test Statistic (BDTS), the resulting distribution involves a weighted sum of independent chi-square variables, where the weights depend on the unknown true density. In finite samples, estimating these weights can be computationally demanding and may lead to size distortions. To provide a readily implementable and precisely calibrated inference tool, we employ a parametric bootstrap approach. For testing the hypothesis $H_0:\mu =0$ against $H_1:\mu >0$, the procedure is as follows:

1. 

Compute the robust estimate ${\widehat{\mu }}_{obs}$ from the observed data using the MEWDE. The test statistic is defined as $T_{obs}={\widehat{\mu }}_{obs}$.

2. 

Generate B bootstrap samples $X_1^{*},\dots ,X_B^{*}$ from the null distribution $N(0,1)$.

3. 

For each bootstrap sample, compute the estimate ${\widehat{\mu }}_b^{*}$ and the statistic $T_b^{*}={\widehat{\mu }}_b^{*}$.

4. 

Compute the p-value as ${\displaystyle \frac{1}{B}}{\sum }_{b=1}^{B}I(T_b^{*}\ge T_{obs})$.

This procedure ensures that the test maintains the nominal size α under the null hypothesis while inheriting the robustness properties of the estimator.

6.3. Comparative Power Analysis: Simulation Design and Findings

To explicitly evaluate the performance of the MEWDE against the established density power divergence (MDPDE) and the classical likelihood ratio test (LRT), we conducted a matched-efficiency power simulation. The data were generated from a contaminated normal model $(1-ϵ)N(\mu ,1)+ϵN(3,1)$, where $\mu$ represents the signal strength varying from $0.0$ to $1.0$. We fixed the sample size at $n=100$ and considered contamination levels $ϵ\in \{0,0.05,0.10,0.20\}$. A critical aspect of this comparison is ensuring that the robust estimators are tuned fairly. We selected a standard robust tuning parameter for the MDPDE ($\alpha =0.30$) and determined the corresponding MEWDE parameter ($\beta =0.10$) that yields an identical Mean Squared Error (MSE) under the clean null model ($N(0,1)$). This calibration ensures that any observed differences in power or stability are attributable to the properties of the divergence measures rather than unequal efficiency trade-offs. Figure 8 displays the empirical power curves for the three methods. The results highlight two key findings:

• Efficiency Matching for Clean Data: In the absence of contamination (0% panel), the power curves for the MEWDE and MDPDE are virtually indistinguishable and lie slightly below the optimal LRT. This confirms the success of our calibration strategy and demonstrates that the MEWDE incurs no additional efficiency cost compared to the DPD when the model is correctly specified.
• Superior Robustness of Validity: In the contaminated scenarios, the differentiation between methods becomes clear. The classical LRT exhibits catastrophic size distortion, rejecting the null nearly 100% of the time at $\mu =0$. The MDPDE, while significantly more robust than the LRT, still suffers from observable size inflation (higher Type I error) in the presence of outliers. The MEWDE uniquely maintains much better control of the nominal size (≈0.05) at the null across all contamination levels. This confirms that the MEWDE offers superior robustness of validity, successfully decoupling inference from contamination where even the standard robust competitor (DPD) shows sensitivity.

6.4. An Example: Testing of Hypothesis for the Shoshoni Data

The Shoshoni data, considered earlier in Section 5.1, are assumed to come from a $N(\mu ,{\sigma }^2)$ distribution with both location and scale parameters being unknown. Following the benchmark analysis of this dataset by Hettmansperger and McKean [21], we consider the null hypothesis $H_0:\mu =0.618$ versus $H_0:\mu \ne 0.618$. We adopt this specific null value to ensure our robust testing results are directly comparable to the findings reported in the original analysis. Ref. [21] notes that if we were to implement the outlier-sensitive likelihood ratio test for the hypothesis

$$H_0:\mu =0.618\mathrm{versus}{H}_1:\mu \ne 0.618,$$

(22)

we would get a p-value of 0.053, which is at the borderline of significance at 5% level. On the other hand, the non-parametric one-sample sign test returns an entirely insignificant p value of 0.823. It would be interesting to investigate the performance of the robust test statistic presented in Section 6.2 in this context. In particular, we focus on the test statistic obtained by setting $B_1=B_2=B$, where B is the function corresponding to the exponentially weighted divergence, as defined in Equation (5). We term the test statistic so obtained as the exponentially weighted divergence test statistic (denoted by EWDTS($\beta$)) and compute it for varying values of $\beta >0$. For $\beta \to 0$, the test resembles the standard likelihood ratio test and returns a p-value slightly exceeding the $0.05$ threshold, as reported by [21]. As $\beta$ increases, the p-value quickly becomes highly insignificant, indicating that the borderline significance under the likelihood based methods is driven by the outliers. Note that the p-value for the t-test statistic for the data with three outliers removed is $0.329$, which conforms to the p-values of the EWDTS for moderately large values of $\beta$. The graph of the p-values (Figure 9) demonstrates the outlier stability of EWDTS for large values of $\beta$.

7. Closing Remarks

In this paper, we have presented an estimator based on a sub-class of density-based Bregman divergences, which is seen to be outperforming the existing standard (i.e., the DPD-based estimator). We have shown several asymptotic and distributional properties of the proposed estimator, both in the context of i.i.d data as well as independent and non-homogeneous data. A special case of linear regression (both simple and multiple) has been explored in the context of real data. We have also discussed “judicial” choice(s) of the tuning parameter which, when chosen properly, yields highly robust and efficient estimators which can often dominate the MDPDE. We have also considered an hypothesis testing strategy for parametric models which may serve as robust alternatives to the classical likelihood ratio and other likelihood-based tests. As we have noted, when the weight function generated by EWD converges to 1 as its argument, the value of the density function increases. We feel that this is the more balanced way for weighting the observations, rather than the weighing provided by the DPD, where the weights increase indefinitely with increase in the value of the density.

7.1. Practical Recommendations for Practitioners

Based on the theoretical properties and empirical results presented in this work, we offer the following guidelines for practitioners choosing between the proposed MEWDE and existing alternatives like the MDPDE:

• Hypothesis Testing and Size Control: The MEWDE is strictly preferable in hypothesis testing scenarios where controlling the Type I error rate is critical (e.g., regulatory clinical trials). As demonstrated in the power analysis (Figure 8), the MEWDE maintains the correct nominal size (≈0.05) even under heavy contamination, whereas the MDPDE can suffer from size inflation.
• Interpretation of Weights: In settings involving high-leverage points, the MEWDE offers a safer interpretation of “downweighting.” The exponentially decaying weight function $w\left(t\right)=1-exp(-t/\beta )$ is strictly bounded in $[0,1]$, ensuring that no observation—regardless of how well it fits the model—can exert undue influence. This contrasts with power-divergence weights, which can theoretically grow unbounded depending on the model parameterization.
• Implementation in Complex Models: For practitioners concerned with the integration burden of the asymptotic null distribution, we recommend the parametric bootstrap approach detailed in Section 6.2. This method is computationally stable, integrates easily with existing statistical software (requiring only a standard optimization routine), and avoids the complexity of estimating the eigenvalues of the weighted chi-square distribution.

7.2. Diagnostics for Model Failure

Since the theoretical guarantees of the MEWDE rely on the compatibility between the assumed model family and the bulk of the data, practitioners should monitor the resulting robust weights ($w_i$) as a post hoc diagnostic. In a successful fit, weights should cluster near 1 for inliers and near 0 for outliers. A specific failure mode occurs in “heavy-tailed” settings (e.g., Cauchy data modeled as normal) where the tails of the data far exceed the support of the model. In this case, the estimator may attempt to flatten the density to accommodate the spread, causing the estimated density values $f_{\theta }\left(x_i\right)$ to become negligible for all points. This results in a weight collapse, where the contributing weights $w_i\approx 0$. Practitioners observing consistently low weights across the sample should interpret this as a signal that the chosen parametric family is insufficiently heavy-tailed for the data, necessitating either a different model family (e.g., t-distribution) or a breakdown-point estimator such as LTS.

7.3. Limitations

While the MEWDE demonstrates strong robustness against outliers and contamination within the support of the distribution, it is important to acknowledge certain theoretical limitations. First, like most M-estimators derived from the exponential family, the validity of the asymptotic theory (specifically Assumptions 5–7 relies on the integrability of the weighted score function). In settings with extremely heavy-tailed data (e.g., Cauchy-distributed errors) or severe leverage points where the underlying moments do not exist, the required integrals may diverge. In such pathological cases, high-breakdown methods such as Least Trimmed Squares (LTS) or MM-estimators may be more appropriate, albeit at the cost of efficiency. Second, the current formulation of the EWD focuses on the standard $p<n$ regime; as noted earlier, extending this framework to high-dimensional settings ($p\gg n$) requires the incorporation of sparsity-inducing penalties, which is a subject of our ongoing research.

7.4. Future Work

It may also be mentioned that the proposal based on the EWD has the potential to be useful in all the situations where the DPD has been successfully applied, such as generalized linear models, survival analysis and Bayesian inference, to name a few. We hope to pursue all of these in our future research. A systematic comparison of alternative bounded weight functions within the Bregman framework is another interesting direction for future research, but lies beyond the scope of the present paper. In case of hypothesis testing, we have only investigated the analogues of the likelihood ratio tests. Other Wald-type tests based on the EWD should also be studied which are likely to have simpler asymptotic null distributions compared to that in Equation (21). The DPD-based Wald-type test has been extensively used in the literature, and comparisons with EWD-based tests will be interesting. We also hope to refine the tuning parameter selection strategy using the recently developed method of [26].