A Robust Version of the Empirical Likelihood Estimator

Abstract

In this paper, we introduce a robust version of the empirical likelihood estimator for semiparametric moment condition models. This estimator is obtained by minimizing the modified Kullback–Leibler divergence, in its dual form, using truncated orthogonality functions. We prove the robustness and the consistency of the new estimator. The performance of the robust empirical likelihood estimator is illustrated through examples based on Monte Carlo simulations.

1. Introduction

A moment condition model is a family ${\mathcal{M}}^{\left(1\right)}$ of probability measures (p.m.), all defined on the same measurable space $({\mathbb{R}}^m,\mathcal{B}\left({\mathbb{R}}^m\right))$, such that (s.t.)

$${\int }_{{\mathbb{R}}^m}g(x,\theta )dQ\left(x\right)=0\mathrm{for}\mathrm{all}Q\in {\mathcal{M}}^{\left(1\right)},$$

where $\mathcal{B}\left({\mathbb{R}}^m\right)$ is the Borel $\sigma$-field. The parameter of interest $\theta$ belongs to a compact set $\mathsf{\Theta }\subset {\mathbb{R}}^d$ and the function $g:={(g_1,\dots ,g_{\ell })}^{\top }$, with $\ell \ge d$, is defined on ${\mathbb{R}}^m\times \mathsf{\Theta }$, each component $g_i$ being real valued function. Denote by $M^{\left(1\right)}$ the set of all probability measures on $({\mathbb{R}}^m,\mathcal{B}\left({\mathbb{R}}^m\right))$, and for each $\theta \in \mathsf{\Theta }$ define

$${\mathcal{M}}_{\theta }^{\left(1\right)}:=\left\{Q\in M^{\left(1\right)}\mathrm{s}.\mathrm{t}.{\int }_{{\mathbb{R}}^m}g(x,\theta )dQ\left(x\right)=0\right\},$$

so that

$${\mathcal{M}}^{\left(1\right)}=\bigcup _{\theta \in \mathsf{\Theta }}{\mathcal{M}}_{\theta }^{\left(1\right)}.$$

(1)

Let $X_1,\dots ,X_n$ be an i.i.d. sample with unknown p.m. $P_0$. We assume that the equation $\int g(x,\theta )d{P}_0\left(x\right)=0$ has a unique solution (in $\theta$), which will be denoted ${\theta }_0$. We consider the estimation problem of ${\theta }_0$ from the data $X_1,\dots ,X_n$. The traditional way of estimating the parameter ${\theta }_0$ is given by the Generalized Method of Moments (GMM) [1]. The GMM estimators are consistent and have asymptotically normal distribution. Despite their desirable asymptotic properties, the finite sample performance of the GMM estimators is not satisfactory. Some other alternative methods have been proposed in the literature. The Continuous Updating (CU) estimator [2], the Empirical Likelihood (EL) estimator [3,4,5], and the Exponential Tilting (ET) [6], are three of the most known examples. Imbens [7] showed that EL and ET estimators are characterized by lower bias than GMM in nonlinear models. Newey and Smith [8] studied the theoretical properties of EL, ET, and CU estimators, by including them into the Generalized Empirical Likelihood (GEL) family of estimators, and showed that all GEL estimators are characterized by lower asymptotic bias than GMM. The information and entropy econometric (IEE) techniques have been proposed in order to improve the finite sample performance of the GMM estimators and tests [4,6]. Ronchetti and Trojani [9] and Lo and Ronchetti [10] have proposed robust alternatives for GMM estimators and tests, respectively, for the IEE techniques, keeping also the finite sample accuracy. Felipe et al. [11] proposed empirical divergence test statistics based on exponentially tilted empirical likelihood estimator with good robustness properties. Broniatowski and Keziou [12] proposed a general approach for estimation and testing in moment condition models, which includes some of the above-mentioned methods. This approach, based on minimizing divergences with their dual forms, allows the asymptotic study of the estimators (called minimum empirical divergence estimators) and of the associated test statistics, both under the model and under mis-specification of the model. The approach based on divergences and duality was firstly considered in the case of parametric models, for example, in [13,14,15]. Applications of the minimum dual divergence estimators in model selection problems are considered in [16].

The EL paradigm enters as a special case of the general methodology from Broniatowski and Keziou [12], namely, when using the modified Kullback–Leibler divergence. Although the EL estimator is preferable to other estimators due to higher-order asymptotic properties, these properties are valid only in the case of the correct specification of the moment conditions. On the other hand, when the support of the p.m. corresponding to the model and the orthogonality functions are not bounded, the EL estimator may cease to be root n consistent (Schennach [17]) under mis-specification. It is a known fact that the EL estimator for moment condition models is not robust. This fact is also justified by the results from [18], where it is shown that the influence function of a minimum dual divergence estimator, and particularly that of the EL estimator, is linear related to the orthogonality function g corresponding to the model. Thus, the influence function of the EL estimator is bounded if and only if the function g corresponding to the underlying model is bounded. Hence, the EL estimator is usually not robust, since the function g could be unbounded on the observation. For this reason, in practice, the classical EL estimator, as well as the minimum dual divergence estimators and also GMM estimators, is unstable even under small deviations from the assumed model.

As examples in this context, we mention models for which the orthogonality functions are unbounded [9]. The autoregressive models with heteroscedastic errors [19] can be written under the form of moment condition models, but the orthogonality functions defining the orthogonality conditions are unbounded. Moreover, the nonlinear empirical asset pricing models [20] can be written under the form of moment condition models and have natural orthogonality conditions (given by the given asset pricing equations), which are given by unbounded orthogonality functions. We also recall the following classical example, which is used in the last section of the paper, in the Monte Carlo simulation study.

Example 1

([3] p. 302). Consider a random variable X with unbounded support ($\mathbb{R}$ or ${\mathbb{R}}_{+}$ for instance). Let $\mathbb{E}\left(X\right)=\theta$, and assume that $\mathbb{E}\left(X^2\right)=h\left(\theta \right)$, with $h(·)$ being a known function. The aim is to estimate the parameter θ using an i.i.d. sample $X_1,\dots ,X_n$ of X. The information on the probability distribution $P_0$ of X can be expressed in the context of model (1), with $d=1$ and $\ell =2$, by taking $g(x,\theta )=:{(g_1(x,\theta ),g_2(x,\theta ))}^{\top }={(x-\theta ,x^2-h\left(\theta \right))}^{\top }$. One can see that the orthogonality functions $g_1(·,\theta )$ and $g_2(·,\theta )$ are unbounded (with respect to x).

For such models, the lack of robustness of the EL estimator, as well as the lack of robustness of other classical estimators, represents the motivation to study some robust alternatives.

In the present paper, we propose a locally robust version of the EL estimator for moment condition models. Locally robust in the sense that the functional associated with this estimator is locally approximated by the means of the influence function, and then, the boundedness of the influence function will imply the fact that, in the neighborhood of the model, the asymptotic bias of the estimator cannot become arbitrarily large (see [21]). The new estimator is defined by minimization of an empirical version of the modified Kullback–Leibler divergence in dual form, using truncated orthogonality functions. This leads to a robust EL estimate. Moreover, we prove the consistency of this estimator, Finally, we present an example based on Monte Carlo simulations illustrating the performance of the robust EL estimator in the case of contaminated data.

2. A Robust Version of the Empirical Likelihood Estimator

2.1. Statistical Divergences

Let $\phi$ be a convex function on $\mathbb{R}$ onto $\left[0,+\infty \right]$ valued, satisfying $\phi \left(1\right)=0$. Let P be some p.m. on the measurable space $({\mathbb{R}}^m,\mathcal{B}\left({\mathbb{R}}^m\right))$. For any signed finite measure Q, on the same measurable space $({\mathbb{R}}^m,\mathcal{B}\left({\mathbb{R}}^m\right))$, absolutely continuous with respect to P, the $\phi$-divergence (sometimes we simply say divergence) between Q and P is defined by

$$D_{\phi }(Q,P):={\int }_{{\mathbb{R}}^m}\phi \left(\frac{dQ}{dP}\left(x\right)\right)dP\left(x\right),$$

(2)

where $\frac{dQ}{dP}$ is the Radon–Nikodym derivative. When Q is not absolutely continuous with respect to $P,$ we set $D_{\phi }(Q,P)=+\infty$. This definition extends the one given in [22] for divergences between p.m.’s. A known class of divergences between p.m.’s is the class of Cressie–Read divergences, introduced in [23] and defined by the functions

$${\phi }_{\gamma }:x\in {\mathbb{R}}_{+}\mapsto {\phi }_{\gamma }\left(x\right):=\frac{x^{\gamma }-\gamma x+\gamma -1}{\gamma (\gamma -1)},$$

(3)

for $\gamma \in \mathbb{R}\backslash \{0,1\}$, ${\phi }_0\left(x\right):=-logx+x-1$ and ${\phi }_1\left(x\right):=xlogx-x+1$. For any $\gamma \in \mathbb{R}$, if ${\phi }_{\gamma }\left(0\right)$ is not defined, we set ${\phi }_{\gamma }\left(0\right):={lim}_{x\to 0^{+}}{\phi }_{\gamma }\left(x\right)$, which may be finite or infinite. The Kullback–Leibler divergence ($KL$) is associated to ${\phi }_1$, the modified Kullback–Leibler ($K{L}_m$) to ${\phi }_0$, the ${\chi }^2$ divergence to ${\phi }_2$, the modified ${\chi }^2$ divergence (${\chi }_m^2$) to ${\phi }_{-1}$ and the Hellinger (H) distance to ${\phi }_{1/2}$. The $\phi$-divergence between some set $\mathsf{\Omega }$ of probability measures and a probability measure P is defined by

$$D_{\phi }(\mathsf{\Omega },P):=\underset{Q\in \mathsf{\Omega }}{inf}{D}_{\phi }(Q,P).$$

(4)

2.2. Definition of the Estimator

We consider a reference identifiable model $\left\{P_{\theta };\theta \in \mathsf{\Theta }\right\}$ of probability measures such that, for each $\theta \in \mathsf{\Theta }$, $P_{\theta }\in {\mathcal{M}}_{\theta }$, which means that ${\int }_{{\mathbb{R}}^m}g(x,\theta )d{P}_{\theta }\left(x\right)=0$, and assume that $\theta$ is the unique solution of the equation. We assume that the p.m. $P_0$ of the data corresponding to the true unknown value ${\theta }_0$ of the parameter to be estimated belongs to this reference model. The reference model will be associated with the truncated orthogonality function that will be used to define the robust version of the EL estimator of the parameter ${\theta }_0$. We will use the notation $\parallel ·\parallel$ for the Euclidean norm. Similarly as in [9], using the reference model $\left\{P_{\theta };\theta \in \mathsf{\Theta }\right\}$, we define the function $g^c:{\mathbb{R}}^m\times \mathsf{\Theta }\to {\mathbb{R}}^{\ell }$,

$$g^c(x,\theta ):=H_c\left(A_{\theta }\left[g(x,\theta )-{\tau }_{\theta }\right]\right),$$

(5)

where $H_c:{\mathbb{R}}^{\ell }\to {\mathbb{R}}^{\ell }$ is the Huber’s function

$$H_c\left(y\right):=\left\{\begin{array}{c}y·min\left(1,\frac{c}{\parallel y\parallel }\right)\mathrm{if}y\ne 0,\hfill \\ 0\mathrm{if}y=0,\hfill \end{array}\right.$$

(6)

and $A_{\theta }$, ${\tau }_{\theta }$ are, respectively, $\ell \times \ell$-matrix and ℓ-vector, defined as the solutions of the system of implicit equations

$$\left\{\begin{array}{c}\int g^c(x,\theta )d{P}_{\theta }\left(x\right)=0,\hfill \\ \int g^c(x,\theta )g^c{(x,\theta )}^{\top }d{P}_0\left(x\right)=I_{\ell },\hfill \end{array}\right.$$

(7)

where $I_{\ell }$ is the $\ell \times \ell$ identity matrix and $c>0$ is a given positive constant. Therefore, we have $\parallel g^c(x,\theta )\parallel \le c$, for all x and $\theta$. We also use the function

$$h^c(x,\theta ,A,\tau ):=H_c\left(A\left[g\right(x,\theta )-\tau ]\right),$$

(8)

when needed to work with the dependence on the matrix A and on the vector $\tau$. Therefore,

$$g^c(x,\theta )=h^c(x,\theta ,A_{\theta },{\tau }_{\theta }).$$

(9)

For given $P_{\theta }$ from the reference model, the triplet $(\theta ,A_{\theta },{\tau }_{\theta })$ is the unique solution of the system

$$\left\{\begin{array}{c}\int g(x,\theta )d{P}_{\theta }\left(x\right)=0,\hfill \\ \int g^c(x,\theta )d{P}_{\theta }\left(x\right)=0,\hfill \\ \int g^c(x,\theta )g^c{(x,\theta )}^{\top }d{P}_0\left(x\right)=I_{\ell }\hfill \end{array}\right.$$

(10)

see [9], p. 48.

Consider the estimating problem of the triplet $({\theta }_0,A_{{\theta }_0},{\tau }_{{\theta }_0})$ on the basis of a sample $X_1,\cdots ,X_n\sim P_0$, $P_0\in {\mathcal{M}}_{{\theta }_0}$. For each $\theta \in \mathsf{\Theta }$, using the p.m. $P_{\theta }$ from the reference model, we define ${\widehat{A}}_{\theta }$ and ${\widehat{\tau }}_{\theta }$, solutions of the system

$$\left\{\begin{array}{c}\int h^c(x,\theta ,{\widehat{A}}_{\theta },{\widehat{\tau }}_{\theta })d{P}_{\theta }\left(x\right)=0,\hfill \\ \int h^c(x,\theta ,{\widehat{A}}_{\theta },{\widehat{\tau }}_{\theta })h^c{(x,\theta ,{\widehat{A}}_{\theta },{\widehat{\tau }}_{\theta })}^{\top }d{P}_n\left(x\right)=I_{\ell },\hfill \end{array}\right.$$

(11)

where $P_n(·)$ is the empirical measure associated with the sample,

$$P_n(·):=\frac{1}{n}\sum _{i=1}^n{\delta }_{X_i}(·),$$

with ${\delta }_x(·)$ being the Dirac measure at the point x, for any x. We denote

$$g_n^c(x,\theta ):=h^c(x,\theta ,{\widehat{A}}_{\theta },{\widehat{\tau }}_{\theta })=H_c\left({\widehat{A}}_{\theta }[g(x,\theta )-{\widehat{\tau }}_{\theta }]\right).$$

(12)

Note that $g_n^c(x,\theta )$ depends on both the data and the reference probability $P_{\theta }$. We now consider the moment condition model associated to the function $g_n^c(x,\theta )$, namely,

$${\mathcal{M}}^{c,n}:=\bigcup _{\theta \in \mathsf{\Theta }}{\mathcal{M}}_{\theta }^{c,n},$$

(13)

where

$${\mathcal{M}}_{\theta }^{c,n}:=\left\{Q\in M^{\left(1\right)}\mathrm{s}.\mathrm{t}.\int g_n^c(x,\theta )dQ\left(x\right)=0\right\},\forall \theta \in \mathsf{\Theta }.$$

(14)

The p.m. $P_0$ belongs to ${\mathcal{M}}_{{\theta }_0}^{c,n}$.

In what follows, we consider the modified Kullback–Leibler divergence, which corresponds to the strictly convex function $\phi \left(x\right):=-logx+x-1$, if $x>0$, respectively $\phi \left(x\right):=+\infty$, if $x\le 0$. The convex conjugate, called also the Fenchel–Legendre transform, of any function $f:x\in \mathbb{R}\mapsto f\left(x\right)\in \overline{\mathbb{R}}$, is the function defined by $f^{*}\left(u\right):={sup}_{x\in \mathbb{R}}\{ux-f\left(x\right)\}$, for all $u\in \mathbb{R}$. A straightforward calculus shows that the convex conjugate of the convex function $\phi$, denote it $\psi$, is given by $\psi \left(u\right)=-log(1-u)$ if $u<1$, respectively $\psi \left(u\right)=+\infty$, if $u\ge 1$. For given $\theta \in \mathsf{\Theta }$, we define the set

$${\overline{\Lambda }}_{\theta }^{c,n}\left(P_0\right):=\left\{\overline{t}:={(t_0,t_1,\dots ,t_{\ell })}^{\top }\in {\mathbb{R}}^{1+\ell }\mathrm{s}.\mathrm{t}.\int \left|\psi \left({\overline{t}}^{\top }{\overline{g}}_n^c(x,\theta )\right)\right|d{P}_0\left(x\right)<\infty \right\},$$

(15)

where ${\overline{g}}_n^c:={({\mathbf{1}}_{{\mathbb{R}}^m},{g_n^c}^{\top })}^{\top }$. We denote ${\overline{\Lambda }}_{\theta }^{c,n}:={\overline{\Lambda }}_{\theta }^{c,n}\left(P_0\right)$ and ${\overline{\Lambda }}_{\theta }^{c,n,n}:={\overline{\Lambda }}_{\theta }^{c,n}\left(P_n\right).$

Since $g_n^c(x,\theta )$ is bounded (with respect to x), on the basis of Theorem 1.1 in [24] and Proposition 4.2 in [12], the following dual representation of $K{L}_m$-divergence holds

$$K{L}_m({\mathcal{M}}_{\theta }^{c,n},P_0)=\underset{\overline{t}\in {\overline{\Lambda }}_{\theta }^{c,n}}{sup}\int m_n^c(x,\theta ,\overline{t})d{P}_0\left(x\right),$$

(16)

where

$$m_n^c(x,\theta ,\overline{t}):=t_0-\psi \left({\overline{t}}^{\top }{\overline{g}}_n^c(x,\theta )\right)=t_0+log(1-{\overline{t}}^{\top }{\overline{g}}_n^c(x,\theta )),$$

(17)

and the supremum in (16) is reached, provided that $K{L}_m({\mathcal{M}}_{\theta }^{c,n},P_0)$ is finite. Note that $m_n^c(x,\theta ,\overline{t})$ depends on the reference p.m. $P_{\theta }$, since $g_n^c(x,\theta )$ depends on $P_{\theta }$. W denote then ${\overline{t}}_{\theta }^{c,n}={\overline{t}}_{\theta }^{c,n}\left(P_0\right)$ any vector such that

$${\overline{t}}_{\theta }^{c,n}:=\underset{\overline{t}\in {\overline{\Lambda }}_{\theta }^{c,n}}{argsup}\int m_n^c(x,\theta ,\overline{t})d{P}_0\left(x\right).$$

(18)

Furthermore, according to Proposition 4.2 from [12], for each $\theta \in \mathsf{\Theta }$, the condition

$$P_0\left(\{x\in {\mathbb{R}}^m\mathrm{s}.\mathrm{t}.{\overline{t}}^{\top }{\overline{g}}_n^c(x,\theta )\ne 0\}\right)>0,\mathrm{for}\mathrm{all}\overline{t}\in {\mathbb{R}}^{1+\ell }\backslash \left\{0\right\},$$

(19)

ensures that ${\overline{t}}_{\theta }^{c,n}$, defined as solution of optimization problem (18), is unique. Notice that the linear independence of the functions ${\mathbf{1}}_{{\mathbb{R}}^m},g_{n,1}^c(·,\theta ),\cdots ,g_{n,\ell }^c(·,\theta )$ implies condition (19) whenever $P_0$ is not degenerate.

Moreover, using again Proposition 4.2 and Remark 4.4 from [12], for each $\theta \in \mathsf{\Theta }$, one can show that the first component of the optimal solution ${\overline{t}}_{\theta }^{c,n}$ in (18) equals to zero. One can then omit the first component of the vector $\overline{t}$ in displays ((15)–(18)). Therefore, they will be replaced by

$${\Lambda }_{\theta }^{c,n}\left(P_0\right):=\left\{t:={(t_1,\dots ,t_{\ell })}^{\top }\in {\mathbb{R}}^{\ell }\mathrm{s}.\mathrm{t}.\int \left|\psi \left(t^{\top }{g}_n^c(x,\theta )\right)\right|d{P}_0\left(x\right)<\infty \right\},$$

(20)

$$K{L}_m({\mathcal{M}}_{\theta }^{c,n},P_0)=\underset{t\in {\Lambda }_{\theta }^{c,n}}{sup}\int m_n^c(x,\theta ,t)d{P}_0\left(x\right),$$

(21)

where

$$m_n^c(x,\theta ,t):=-\psi \left(t^{\top }{g}_n^c(x,\theta )\right)=log(1-t^{\top }{g}_n^c(x,\theta ))$$

(22)

and

$$t_{\theta }^{c,n}:=\underset{t\in {\Lambda }_{\theta }^{c,n}}{argsup}\int m_n^c(x,\theta ,t)d{P}_0\left(x\right).$$

(23)

Denote

$${\Lambda }_{\theta }^{c,n,n}:={\Lambda }_{\theta }^{c,n}\left(P_n\right)=\left\{t:={(t_1,\dots ,t_{\ell })}^{\top }\in {\mathbb{R}}^{\ell }\mathrm{s}.\mathrm{t}.\frac{1}{n}\sum _{i=1}^n\left|log(1-\sum _{j=1}^{\ell }{t}_j{g}_{n,j}^c(X_i,\theta ))\right|<\infty \right\}.$$

In view of relation (23), a natural estimator of $t_{\theta }^{c,n}$, is defined by

$$\widehat{t_{\theta }^c}:=\underset{t\in {\Lambda }_{\theta }^{c,n,n}}{argsup}\int m_n^c(x,\theta ,t)d{P}_n\left(x\right).$$

(24)

Then, a “dual” plug-in estimator of the modified Kullback–Leibler divergence, between ${\mathcal{M}}_{\theta }^{c,n}$ and $P_0$, can be defined by

$$\begin{array}{ccc}\hfill \widehat{K{L}_m}({\mathcal{M}}_{\theta }^{c,n},P_0)& :=& \underset{t\in {\Lambda }_{\theta }^{c,n,n}}{sup}\int m_n^c(x,\theta ,t)d{P}_n\left(x\right)\hfill \\ & =& \underset{(t_1,\dots ,t_{\ell })\in {\Lambda }_{\theta }^{c,n,n}}{sup}\left\{\int log\left(1-\sum _{j=1}^{\ell }{t}_j{g}_{n,j}^c(x,\theta )\right)d{P}_n\left(x\right)\right\}\hfill \\ & =& \underset{(t_1,\dots ,t_{\ell })\in {\mathbb{R}}^{\ell }}{sup}\left\{\frac{1}{n}\sum _{i=1}^n\overline{log}\left(1-\sum _{j=1}^{\ell }{t}_j{g}_{n,j}^c(X_i,\theta )\right)\right\},\hfill \end{array}$$

(25)

where $\overline{log}(·)$ is the extended logarithm function, i.e., the function defined by $\overline{log}\left(u\right)=log\left(u\right)$ if $u>0$, and $\overline{log}\left(u\right)=-\infty$ if $u\le 0$, for any $u\in \mathbb{R}.$ Finally, we define the following estimator of ${\theta }_0$

$$\begin{array}{ccc}\hfill \widehat{{\theta }^c}& :=& \underset{\theta \in \mathsf{\Theta }}{arginf}\underset{t\in {\Lambda }_{\theta }^{c,n,n}}{sup}\int m_n^c(x,\theta ,t)d{P}_n\left(x\right)\hfill \\ & =& \underset{\theta \in \mathsf{\Theta }}{arginf}\underset{(t_1,\dots ,t_{\ell })\in {\mathbb{R}}^{\ell }}{sup}\left\{\frac{1}{n}\sum _{i=1}^n\overline{log}\left(1-\sum _{j=1}^{\ell }{t}_j{g}_{n,j}^c(X_i,\theta )\right)\right\},\hfill \end{array}$$

(26)

which can be seen as a “robust” version of the well-known EL estimator.

Recall that the EL estimator can be written as (see, e.g., [5])

$$\widehat{\theta }=\underset{\theta \in \mathsf{\Theta }}{arginf}\underset{(t_1,\dots ,t_{\ell })\in {\mathbb{R}}^{\ell }}{sup}\left\{\frac{1}{n}\sum _{i=1}^n\overline{log}\left(1-\sum _{j=1}^{\ell }{t}_j{g}_j(X_i,\theta )\right)\right\}.$$

(27)

For establishing asymptotic properties of the proposed estimators, we need the following additional notations. Consider the moment condition model associated with the truncated function

$$g^c(x,\theta )=H_c\left(A_{\theta }[g(x,\theta )-{\tau }_{\theta }]\right),$$

where $A_{\theta }$ and ${\tau }_{\theta }$ are the solution to the system (7). Note that $g^c(x,\theta )$ depends only on the reference model $P_{\theta }$ and not on the data. This model is defined by

$${\mathcal{M}}^c:=\bigcup _{\theta \in \mathsf{\Theta }}{\mathcal{M}}_{\theta }^c,$$

(28)

where

$${\mathcal{M}}_{\theta }^c:=\left\{Q\in M^{\left(1\right)}\mathrm{s}.\mathrm{t}.\int g^c(x,\theta )dQ\left(x\right)=0\right\},\forall \theta \in \mathsf{\Theta }.$$

(29)

Let

$${\Lambda }_{\theta }^c\left(P_0\right):=\left\{t:={(t_1,\dots ,t_{\ell })}^{\top }\in {\mathbb{R}}^{\ell }\mathrm{s}.\mathrm{t}.\int \left|\psi \left(t^{\top }{g}^c(x,\theta )\right)\right|d{P}_0\left(x\right)<\infty \right\}.$$

(30)

Therefore, as above, we have the following dual representation for $K{L}_m({\mathcal{M}}_{\theta }^c,P_0)$

$$K{L}_m({\mathcal{M}}_{\theta }^c,P_0)=\underset{t\in {\Lambda }_{\theta }^c}{sup}\int m^c(x,\theta ,t)d{P}_0\left(x\right),$$

(31)

where

$$m^c(x,\theta ,t):=-\psi \left(t^{\top }{g}^c(x,\theta )\right)=log(1-t^{\top }{g}^c(x,\theta )),$$

(32)

and the supremum in (31) is reached, provided that $K{L}_m({\mathcal{M}}_{\theta }^c,P_0)$ is finite. Moreover, the supremum in (31) is unique under the following assumption

$$P_0\left(\{x\in {\mathbb{R}}^m\mathrm{s}.\mathrm{t}.{\overline{t}}^{\top }{\overline{g}}^c(x,\theta )\ne 0\}\right)>0,\mathrm{for}\mathrm{all}\overline{t}\in {\mathbb{R}}^{1+\ell }\backslash \left\{0\right\},$$

(33)

which is satisfied if the functions ${\mathbf{1}}_{{\mathbb{R}}^m},g_1^c(·,\theta ),\cdots ,g_{\ell }^c(·,\theta )$ are linearly independent and $P_0$ is not degenerate. We denote then

$$t_{\theta }^c:=\underset{t\in {\Lambda }_{\theta }^c}{argsup}\int m^c(x,\theta ,t)d{P}_0\left(x\right).$$

(34)

Finally, we have

$$\begin{array}{ccc}\hfill {\theta }_0& =& \underset{\theta \in \mathsf{\Theta }}{arginf}K{L}_m({\mathcal{M}}_{\theta }^c,P_0)\hfill \\ & =& \underset{\theta \in \mathsf{\Theta }}{arginf}\underset{t\in {\Lambda }_{\theta }^c}{sup}\int m^c(x,\theta ,t)d{P}_0\left(x\right)=\underset{\theta \in \mathsf{\Theta }}{arginf}\int m^c(x,\theta ,t_{\theta }^c)d{P}_0\left(x\right).\hfill \end{array}$$

(35)

We also use the function

$$n^c(x,\theta ,t,A,\tau ):=log(1-t^{\top }{h}^c(x,\theta ,A,\tau )),$$

(36)

when needed to work with the dependence on matrix A and on vector $\tau$. We have, then, with the above notation, $m^c(x,\theta ,t)=n^c(x,\theta ,t,A_{\theta },{\tau }_{\theta }),$ where $A_{\theta }$ and ${\tau }_{\theta }$ are the solution of the system of Equation (7).

2.3. Robustness Property

In order to prove the robustness of the estimator $\widehat{{\theta }^c}$, we use the following well-known tools from the theory of robust statistics; see, e.g., [21]. A functional T, defined on a set of probability measures and parameter space valued, is called a statistical functional associated with an estimator $\widehat{\theta }$ of the parameter $\theta$ from the model $P_{\theta }$, if $\widehat{\theta }=T\left(P_n\right)$. The influence function of T at $P_{\theta }$ is defined by

$$\mathrm{IF}(x;T,P_{\theta }):={\left.\frac{\partial T\left({\tilde{P}}_{\epsilon x}\right)}{\partial \epsilon }\right|}_{\epsilon =0},$$

where ${\tilde{P}}_{\epsilon x}:=(1-\epsilon )P_{\theta }+\epsilon {\delta }_x$. A natural robustness requirement on the statistical functional corresponding to the estimator is the boundedness of its influence function.

The statistical functional corresponding to the estimator $\widehat{{\theta }^c}$ given by (26), is defined by

$$T^c\left(P\right):=\underset{\theta \in \mathsf{\Theta }}{arginf}\underset{t\in {\Lambda }_{\theta }^c\left(P\right)}{sup}\int m^c(y,\theta ,t,P)dP\left(y\right),$$

(37)

where

$${\Lambda }_{\theta }^c\left(P\right):=\left\{t\in {\mathbb{R}}^{\ell }\mathrm{s}.\mathrm{t}.\int \left|\psi \left(t^{\top }{g}^c(x,\theta ,P)\right)\right|dP\left(x\right)<\infty \right\},$$

(38)

$g^c(x,\theta ,P):=h^c(x,\theta ,A_{\theta }\left(P\right),{\tau }_{\theta }\left(P\right))$, with $A_{\theta }\left(P\right)$ and ${\tau }_{\theta }\left(P\right)$ solutions of the system

$$\left\{\begin{array}{c}\int h^c(x,\theta ,A_{\theta }\left(P\right),{\tau }_{\theta }\left(P\right))d{P}_{\theta }\left(x\right)=0\hfill \\ \int h^c(x,\theta ,A_{\theta }\left(P\right),{\tau }_{\theta }\left(P\right))h^c{(x,\theta ,A_{\theta }\left(P\right),{\tau }_{\theta }\left(P\right))}^{\top }dP\left(x\right)=I_{\ell }\hfill \end{array}\right.$$

(39)

and

$$m^c(x,\theta ,t,P):=-\psi \left(t^{\top }{g}^c(x,\theta ,P)\right)=log\left(1-t^{\top }{g}^c(x,\theta ,P)\right).$$

(40)

Note that, for a given $\theta$, the function $g^c(x,\theta ,P)$, as well as $m^c(x,\theta ,t,P)$, both depend on the p.m. $P_{\theta }$. In addition, note that $g^c(x,\theta ,P_{\theta })$ coincides with $g^c(x,\theta )$ defined in the preceding section. We denote

$$t_{\theta }^c\left(P\right):=\underset{t\in {\Lambda }_{\theta }^c\left(P\right)}{argsup}\int m^c(y,\theta ,t,P)dP\left(y\right).$$

(41)

Then

$$T^c\left(P\right)=\underset{\theta \in \mathsf{\Theta }}{arginf}\int m^c(y,\theta ,t_{\theta }^c\left(P\right),P)dP\left(y\right).$$

(42)

Proposition 1.

The influence function of the estimator $\widehat{{\theta }^c}$ is given by

$$\begin{array}{ccc}\hfill \mathrm{IF}(x;T^c,P_0)& =& {\left\{{\left[\int \frac{\partial }{\partial \theta }{g}^c(y,{\theta }_0)d{P}_0\left(y\right)\right]}^{\top }\int \frac{\partial }{\partial \theta }{g}^c(y,{\theta }_0)d{P}_0\left(y\right)\right\}}^{-1}·\hfill \\ & & ·{\left[\int \frac{\partial }{\partial \theta }{g}^c(y,{\theta }_0)d{P}_0\left(y\right)\right]}^{\top }{g}^c(x,{\theta }_0).\hfill \end{array}$$

(43)

Proof. 

Using the definitions of $T^c\left(P\right)$ and $t_{\theta }^c\left(P\right)$, we have

$$\begin{array}{ccc}& & \int \frac{\partial }{\partial t}{m}^c(y,\theta ,t_{\theta }^c\left(P\right),P)dP\left(y\right)=0\hfill \end{array}$$

(44)

$$\begin{array}{ccc}& & \int \frac{\partial }{\partial t}{m}^c(y,T^c\left(P\right),t_{T^c\left(P\right)}^c\left(P\right),P)dP\left(y\right)=0\hfill \end{array}$$

(45)

$$\begin{array}{ccc}& & \int \frac{\partial }{\partial \theta }{\left[m^c(y,\theta ,t_{\theta }^c\left(P\right),P)\right]}_{\theta =T^c\left(P\right)}dP\left(y\right)=0.\hfill \end{array}$$

(46)

Using (46), since

$$m^c(y,\theta ,t_{\theta }^c\left(P\right),P)=log\left(1-t_{\theta }^c{\left(P\right)}^{\top }{h}^c(y,\theta ,A_{\theta }\left(P\right),{\tau }_{\theta }\left(P\right))\right),$$

(47)

$T^c\left(P\right)$ is solution of the equation

$${[\frac{\partial }{\partial \theta }{t}_{\theta }^c\left(P\right)]}^{\top }\int \frac{\partial }{\partial t}{m}^c(y,\theta ,t_{\theta }^c\left(P\right),P)dP\left(y\right)-\int \frac{{[\frac{\partial }{\partial \theta }{h}^c(y,\theta ,A_{\theta }\left(P\right),{\tau }_{\theta }\left(P\right))]}^{\top }{t}_{\theta }^c\left(P\right)}{1-t_{\theta }^c{\left(P\right)}^{\top }{h}^c(y,\theta ,A_{\theta }\left(P\right),{\tau }_{\theta }\left(P\right))}dP\left(y\right)=0.$$

(48)

Since the first integral in the above display is 0, according to (44), the above equation simplifies to

$$\int \frac{{\left[\frac{\partial }{\partial \theta }{h}^c(y,\theta ,A_{\theta }\left(P\right),{\tau }_{\theta }\left(P\right))\right]}^{\top }{t}_{\theta }^c\left(P\right)}{1-t_{\theta }^c{\left(P\right)}^{\top }{h}^c(y,\theta ,A_{\theta }\left(P\right),{\tau }_{\theta }\left(P\right))}dP\left(y\right)=0.$$

(49)

By replacing P with the contaminated model ${\tilde{P}}_{\epsilon x}:=(1-\epsilon )P_0+\epsilon {\delta }_x$ in Equation (49) and then derivating with respect to $\epsilon$, the resulting equation, we obtain

$$\int {\left[\frac{\partial }{\partial \theta }{h}^c(y,{\theta }_0,A_{{\theta }_0}\left(P_0\right),{\tau }_{{\theta }_0}\left(P_0\right))\right]}^{\top }d{P}_0\left(y\right)·\frac{\partial }{\partial \epsilon }{\left[t_{T^c\left({\tilde{P}}_{\epsilon x}\right)}^c\left({\tilde{P}}_{\epsilon x}\right)\right]}_{\epsilon =0}=0.$$

(50)

On the other hand,

$$\frac{\partial }{\partial \epsilon }{\left[t_{T^c\left({\tilde{P}}_{\epsilon x}\right)}^c\left({\tilde{P}}_{\epsilon x}\right)\right]}_{\epsilon =0}=\frac{\partial }{\partial \theta }{t}_{{\theta }_0}^c\left(P_0\right)·\mathrm{IF}(x;T^c,P_0)+\mathrm{IF}(x;t_{{\theta }_0}^c,P_0).$$

(51)

Combining (50) and (51), we get

$$\int {\left[\frac{\partial }{\partial \theta }{h}^c(y,{\theta }_0,A_{{\theta }_0}\left(P_0\right),{\tau }_{{\theta }_0}\left(P_0\right))\right]}^{\top }d{P}_0\left(y\right)·\left\{\frac{\partial }{\partial \theta }{t}_{{\theta }_0}^c\left(P_0\right)·\mathrm{IF}(x;T^c,P_0)+\mathrm{IF}(x;t_{{\theta }_0}^c,P_0)\right\}=0.$$

(52)

Differentiating with respect to $\theta$ of (44) leads to

$$\frac{\partial }{\partial \theta }{t}_{{\theta }_0}^c\left(P_0\right)=-{\left[\int \frac{{\partial }^2}{{\partial }^{2}t}{m}^c(y,{\theta }_0,t_{{\theta }_0}^c\left(P_0\right),P_0)d{P}_0\left(y\right)\right]}^{-1}\int \frac{\partial }{\partial \theta \partial t}{m}^c(y,{\theta }_0,t_{{\theta }_0}^c\left(P_0\right),P_0)d{P}_0\left(y\right).$$

(53)

Some simple calculations yield

$$\frac{\partial }{\partial t}{m}^c(y,\theta ,t,P)=-\frac{h^c(y,\theta ,A_{\theta }\left(P\right),{\tau }_{\theta }\left(P\right))}{1-t^{\top }{h}^c(y,\theta ,A_{\theta }\left(P\right),{\tau }_{\theta }\left(P\right))}$$

(54)

$$\frac{{\partial }^2}{{\partial }^{2}t}{m}^c(y,\theta ,t,P)=-\frac{h^c(y,\theta ,A_{\theta }\left(P\right),{\tau }_{\theta }\left(P\right))h^c{(y,\theta ,A_{\theta }\left(P\right),{\tau }_{\theta }\left(P\right))}^{\top }}{{(1-t^{\top }{h}^c(y,\theta ,A_{\theta }\left(P\right),{\tau }_{\theta }\left(P\right)))}^2}$$

and

$$\begin{array}{ccc}\hfill \frac{\partial }{\partial \theta \partial t}{m}^c(y,\theta ,t,P)& =& \frac{-\frac{\partial }{\partial \theta }{h}^c(y,\theta ,A_{\theta }\left(P\right),{\tau }_{\theta }\left(P\right))(1-t^{\top }{h}^c(y,\theta ,A_{\theta }\left(P\right),{\tau }_{\theta }\left(P\right)))}{{(1-t^{\top }{h}^c(y,\theta ,A_{\theta }\left(P\right),{\tau }_{\theta }\left(P\right)))}^2}\hfill \\ & & +\frac{h^c(y,\theta ,A_{\theta }\left(P\right),{\tau }_{\theta }\left(P\right))(1-t^{\top }\frac{\partial }{\partial \theta }{h}^c(y,\theta ,A_{\theta }\left(P\right),{\tau }_{\theta }\left(P\right))}{{(1-t^{\top }{h}^c(y,\theta ,A_{\theta }\left(P\right),{\tau }_{\theta }\left(P\right)))}^2}.\hfill \end{array}$$

Then, we obtain

$$\int \frac{{\partial }^2}{{\partial }^{2}t}{m}^c(y,{\theta }_0,t_{{\theta }_0}^c\left(P_0\right),P_0)d{P}_0\left(y\right)=-I_{\ell }$$

(55)

and

$$\int \frac{\partial }{\partial \theta \partial t}{m}^c(y,{\theta }_0,t_{{\theta }_0}^c\left(P_0\right),P_0)d{P}_0\left(y\right)=-\int \frac{\partial }{\partial \theta }{h}^c(y,{\theta }_0,A_{{\theta }_0}\left(P_0\right),{\tau }_{{\theta }_0}\left(P_0\right))d{P}_0\left(y\right).$$

(56)

By replacing in (53), we obtain

$$\frac{\partial }{\partial \theta }{t}_{{\theta }_0}^c\left(P_0\right)=-\int \frac{\partial }{\partial \theta }{h}^c(y,{\theta }_0,A_{{\theta }_0}\left(P_0\right),{\tau }_{{\theta }_0}\left(P_0\right))d{P}_0\left(y\right).$$

(57)

Using (52) and (57), we obtain

$$\begin{array}{c}-\int {\left[\frac{\partial }{\partial \theta }{h}^c(y,{\theta }_0,A_{{\theta }_0}\left(P_0\right),{\tau }_{{\theta }_0}\left(P_0\right))\right]}^{\top }d{P}_0\left(y\right)\hfill \\ ·\int \frac{\partial }{\partial \theta }{h}^c(y,{\theta }_0,A_{{\theta }_0}\left(P_0\right),{\tau }_{{\theta }_0}\left(P_0\right))d{P}_0\left(y\right)·\mathrm{IF}(x;T^c,P_0)\\ +\int \frac{\partial }{\partial \theta }{h}^c{(y,{\theta }_0,A_{{\theta }_0}\left(P_0\right),{\tau }_{{\theta }_0}\left(P_0\right))}^{\top }d{P}_0\left(y\right)·\mathrm{IF}(x;t_{{\theta }_0}^c,P_0)=0.\end{array}$$

(58)

In order to calculate $\mathrm{IF}(x;t_{\theta }^c,P_0)$, we first write Equation (44) for $P={\tilde{P}}_{\epsilon x}$ and also use (54). Thus, we obtain

$$\begin{array}{c}(1-\epsilon )\int \frac{h^c(y,\theta ,A_{\theta }\left({\tilde{P}}_{\epsilon x}\right),{\tau }_{\theta }\left({\tilde{P}}_{\epsilon x}\right))}{1-t_{\theta }^c{\left({\tilde{P}}_{\epsilon x}\right)}^{\top }{h}^c(y,\theta ,A_{\theta }\left({\tilde{P}}_{\epsilon x}\right),{\tau }_{\theta }\left({\tilde{P}}_{\epsilon x}\right))}d{P}_0\left(y\right)\hfill \\ \hfill +\epsilon \frac{h^c(x,\theta ,A_{\theta }\left({\tilde{P}}_{\epsilon x}\right),{\tau }_{\theta }\left({\tilde{P}}_{\epsilon x}\right))}{1-t_{\theta }^c{\left({\tilde{P}}_{\epsilon x}\right)}^{\top }{h}^c(x,\theta ,A_{\theta }\left({\tilde{P}}_{\epsilon x}\right),{\tau }_{\theta }\left({\tilde{P}}_{\epsilon x}\right))}=0.\end{array}$$

(59)

By differentiation with respect to $\epsilon$, and taking $\theta ={\theta }_0$, we get

$$\begin{array}{c}\int \frac{\partial }{\partial \epsilon }{\left[h^c(y,{\theta }_0,A_{{\theta }_0}\left({\tilde{P}}_{\epsilon x}\right),{\tau }_{{\theta }_0}\left({\tilde{P}}_{\epsilon x}\right))\right]}_{\epsilon =0}d{P}_0\left(y\right)\hfill \\ -\int h^c(y,{\theta }_0,A_{{\theta }_0}\left(P_0\right),{\tau }_{{\theta }_0}\left(P_0\right))h^c{(y,{\theta }_0,A_{{\theta }_0}\left(P_0\right),{\tau }_{{\theta }_0}\left(P_0\right))}^{\top }d{P}_0\left(y\right)\mathrm{IF}(x;t_{{\theta }_0}^c,P_0)\\ \hfill +h^c(x,{\theta }_0,A_{{\theta }_0}\left(P_0\right),{\tau }_{{\theta }_0}\left(P_0\right))=0.\end{array}$$

Consequently,

$$\begin{array}{ccc}\hfill \mathrm{IF}(x;t_{{\theta }_0}^c,P_0)& =& {\left\{\int h^c(y,{\theta }_0,A_{{\theta }_0}\left(P_0\right),{\tau }_{{\theta }_0}\left(P_0\right))h^c{(y,{\theta }_0,A_{{\theta }_0}\left(P_0\right),{\tau }_{{\theta }_0}\left(P_0\right))}^{\top }d{P}_0\left(y\right)\right\}}^{-1}\hfill \\ & & ·h^c(x,{\theta }_0,A_{{\theta }_0}\left(P_0\right),{\tau }_{{\theta }_0}\left(P_0\right))=h^c(x,{\theta }_0,A_{{\theta }_0}\left(P_0\right),{\tau }_{{\theta }_0}\left(P_0\right)).\hfill \end{array}$$

(60)

By combining (58) with (60), we obtain (43). □

Remark 1.

The classical empirical likelihood estimator of the parameter ${\theta }_0$ of the moment condition model can be obtained as a particular case of the class of minimum empirical divergence estimators introduced by Broniatowski and Keziou [12]. Toma [18] proved that, in the case when $P_0$ belongs to the model ${\mathcal{M}}^{\left(1\right)}$, the influence functions for the estimators from this class, so particularly the influence function of the EL estimator, are all of the form

$$\begin{array}{ccc}\hfill \mathrm{IF}(x;T,P_0)& =& -\left\{{\left[\int \frac{\partial }{\partial \theta }g(y,{\theta }_0)d{P}_0\left(y\right)\right]}^{\top }{\left[\int g(y,{\theta }_0)g{(y,{\theta }_0)}^{\top }d{P}_0\left(y\right)\right]}^{-1}\right.\hfill \\ & & {\left.·\left[\int \frac{\partial }{\partial \theta }g(y,{\theta }_0)d{P}_0\left(y\right)\right]\right\}}^{-1}·{\left[\int \frac{\partial }{\partial \theta }g(y,{\theta }_0)d{P}_0\left(y\right)\right]}^{\top }\hfill \\ & & ·{\left[\int g(y,{\theta }_0)g{(y,{\theta }_0)}^{\top }d{P}_0\left(y\right)\right]}^{-1}g(x,{\theta }_0),\hfill \end{array}$$

(61)

irrespective of the used divergence. This influence function also coincides with the influence function of the GMM estimator obtained by Ronchetti and Trojani [9] and is linearly related to the function $g(x,\theta )$ of the model. When the orthogonality function $g(x,\theta )$ is not bounded in x, the minimum empirical divergence estimators, and particularly the EL estimator of ${\theta }_0$, are not robust. For many moment condition models, the orthogonality functions are linear and hence unbounded; therefore, these estimation methods are generally not robust. This is also the case of other known estimators, such as the least squares estimators, the GMM estimators, and the exponential tilting estimator for moment condition models. Instead, for the new estimator defined in the present paper, the influence function is linearly related to the function $g^c(x,\theta )$, which is bounded; therefore, this estimator can be seen as a robust version of the classical EL estimator.

An important feature of the robust version of the EL estimator is that its robustness can be controlled by a positive constant c. This constant appears in the Huber function used. In addition, an advantage of this approach based on using the Huber function is that we can require the bound of influence function of the estimator to be satisfied in a norm that is self-standardized with respect to the covariance matrix of the estimator; this norm measures the influence of the estimator relative to its variability expressed by its covariance matrix. Such an approach is also suitable to induce stable testing procedures (see [9]), which could be useful in future studies regarding robust testing. The robust estimator proposed in the present paper has the self-standardized influence function bounded by the constant c appearing in the Huber function. In a similar manner as in [9], the constant c controls the degree of robustness of the estimator, and in practice, we could take a value close to the lower bound $\sqrt{\ell }$ in order to enforce a maximum amount of robustness. On the other hand, the density power divergences [25] combined with the minimum divergence approach have proved useful for construction of robust estimators in different contexts, in particular for parameter density estimation. Other approaches for robust estimation in regression models is proposed by [26]. Such approaches could be considered in future research studies in order to be adapted to the context of the moment condition models.

2.4. Consistency of the Estimators

In this subsection, we establish consistency of the estimator $\widehat{t_{\theta }^c}$ of $t_{\theta }^c$, for any fixed $\theta \in \mathsf{\Theta }$, and the consistency of the estimator $\widehat{{\theta }^c}$ of ${\theta }_0$. First, for any fixed $\theta \in \mathsf{\Theta }$, we state the consistency of the estimators ${\widehat{A}}_{\theta }$ and ${\widehat{\tau }}_{\theta }$ defined by the system (11).

2.4.1. Consistency of the Estimators ${\widehat{A}}_{\theta }$ and ${\widehat{\tau }}_{\theta }$, for Fixed $\theta \in \mathsf{\Theta }$

The estimators ${\widehat{A}}_{\theta }$ and ${\widehat{\tau }}_{\theta }$, of $A_{\theta }$ and ${\tau }_{\theta }$ defined by the theoretical system of Equation (7), are Z-estimators. We consider the following notations

$$\begin{array}{ccc}\hfill {\mathsf{\Psi }}_1(\theta ,A,\tau )& :=& \int h^c(y,\theta ,A,\tau )d{P}_{\theta }\left(y\right)\hfill \end{array}$$

(62)

$$\begin{array}{ccc}\hfill {\mathsf{\Psi }}_2(x,\theta ,A,\tau )& :=& h^c(x,\theta ,A,\tau )h^c{(x,\theta ,A,\tau )}^{\top }-I_{\ell }\hfill \end{array}$$

(63)

and $\mathsf{\Psi }(x,\theta ,A,\tau ):={({\mathsf{\Psi }}_1{(\theta ,A,\tau )}^{\top },vec{\left({\mathsf{\Psi }}_2(x,\theta ,A,\tau )\right)}^{\top })}^{\top }$, where “$vec(·)$” is the operator that transforms the matrix into vector, by putting all the columns of the matrix one under the other. Notice that, ${\mathsf{\Psi }}_1(\theta ,A,\tau )$ is a constant function with respect to x. With these notations, for a given $\theta \in \mathsf{\Theta }$, the Z-estimators ${\widehat{A}}_{\theta }$ and ${\widehat{\tau }}_{\theta }$ are solutions of the system (see (11)),

$$\int \mathsf{\Psi }(x,\theta ,{\widehat{A}}_{\theta },{\widehat{\tau }}_{\theta })d{P}_n\left(x\right)=0,$$

(64)

and their theoretical counterparts are $A_{\theta }$ and ${\tau }_{\theta }$ solution of the system (see (7)),

$$\int \mathsf{\Psi }(x,\theta ,A_{\theta },{\tau }_{\theta })d{P}_0\left(x\right)=0.$$

(65)

In the rest of the paper, we consider matrix A in its “vec” form, as defined above. This is necessary in order to apply some classical results, for example the uniform weak law of large numbers (UWLLN) or results regarding Z-estimators. Therefore, the argument A of the functions $h^c$, $n^c$ and $\mathsf{\Psi }$ will be in fact $\mathrm{vec}A$. For simplicity, we write A instead of $\mathrm{vec}A$. The same is valid for ${\widehat{A}}_{\theta }$ and $A_{\theta }$.

Assumption 1.

(a) There exists compact neighborhood $N_{\theta }$ of $(A_{\theta },{\tau }_{\theta })$ such that

$$\int \underset{(A,\tau )\in N_{\theta }}{sup}\parallel \mathsf{\Psi }(x,\theta ,A,\tau )\parallel d{P}_0\left(x\right)<\infty ;$$

(66)

(b) for any positive ε, the following condition holds

$$\begin{array}{c}\hfill \underset{(A,\tau )\in M_{\theta }}{inf}∥\int \mathsf{\Psi }(x,\theta ,A,\tau )d{P}_0\left(x\right)∥>0=∥\int \mathsf{\Psi }(x,\theta ,A_{\theta },{\tau }_{\theta })d{P}_0\left(x\right)∥,\end{array}$$

where $M_{\theta }:=\left\{(A,\tau )\mathrm{s}.\mathrm{t}.∥(A,\tau )-(A_{\theta },{\tau }_{\theta })∥>\epsilon \right\}$.

Proposition 2.

For each $\theta \in \mathsf{\Theta }$, under Assumption 1, $({\widehat{A}}_{\theta },{\widehat{\tau }}_{\theta })$ converges in probability to $(A_{\theta },{\tau }_{\theta })$.

Proof. 

Since $(A,\tau )\mapsto \mathsf{\Psi }(x,\theta ,A,\tau )$ is continuous, by the UWLLN, Assumption 1(a) implies

$$\underset{(A,\tau )\in N_{\theta }}{sup}\parallel \int \mathsf{\Psi }(x,\theta ,A,\tau )d{P}_n\left(x\right)-\int \mathsf{\Psi }(x,\theta ,A,\tau )d{P}_0\left(x\right)\parallel \to 0,$$

(67)

in probability. This result, together with Assumption 1(b) ensure the convergence in probability of the estimators ${\widehat{A}}_{\theta }$ and ${\widehat{\tau }}_{\theta }$ toward $A_{\theta }$ and ${\tau }_{\theta }$, respectively. The arguments are the same as those from [27], Theorem 5.9, p. 46. □

2.4.2. Consistency of the Estimator $\widehat{t_{\theta }^c}$ of $t_{\theta }^c$, for Fixed $\theta \in \mathsf{\Theta }$

We state the consistency of the estimators $\widehat{t_{\theta }^c}$ under the following assumptions.

Assumption 2.

(a) $t_{\theta }^c:=\underset{t\in {\Lambda }_{\theta }^c\left(P_0\right)}{argsup}\int m^c(y,\theta ,t)d{P}_0\left(y\right)$ exists, unique and interior point of ${\Lambda }_{\theta }^c\left(P_0\right)$;

(b) there exists a compact neighborhood $N_{t_{\theta }^c}\subset {\Lambda }_{\theta }^c\left(P_0\right)$ of $t_{\theta }^c$ such that $t_{\theta }^c\in \mathrm{int}\left(N_{t_{\theta }^c}\right)$, and there exists compact neighborhood $N_{\theta }$ of $(A_{\theta },{\tau }_{\theta })$ such that

$$\int \underset{t\in N_{t_{\theta }^c},(A,\tau )\in N_{\theta }}{sup}\left|n^c(x,\theta ,t,A,\tau )\right|d{P}_0\left(x\right)<\infty ;$$

(c) there exists compact neighborhood $N_{t_{\theta }^c}$ of $t_{\theta }^c$, and there exists a sequence $B_n=O_P\left(1\right)$, such that, for all $t,t^{\prime }\in N_{t_{\theta }^c}$, it holds

$$\left|\int n^c(x,\theta ,t,{\widehat{A}}_{\theta },{\widehat{\tau }}_{\theta })d{P}_0\left(x\right)-\int n^c(x,\theta ,t^{\prime },{\widehat{A}}_{\theta },{\widehat{\tau }}_{\theta })d{P}_0\left(x\right)\right|\le B_n\parallel t-t^{\prime }\parallel .$$

(68)

Proposition 3.

Under Assumptions 1 and 2, we have

$\left(1\right)$$\widehat{t_{\theta }^c}$ converges in probability to $t_{\theta }^c$;

$\left(2\right)$$\widehat{K{L}_m}({\mathcal{M}}_{\theta }^{c,n},P_0)$ converges in probability to $K{L}_m({\mathcal{M}}_{\theta }^c,P_0)$.

Proof. 

(1) Using Assumption 2(b) and the continuity of the function $n^c(x,\theta ,t,A,\tau )$ with respect to $t,A$, and $\tau$, by the uniform weak law of large numbers (UWLLN), we get

$$\underset{t\in N_{t_{\theta }^c},(A,\tau )\in N_{\theta }}{sup}\left|\int n^c(x,\theta ,t,A,\tau )d{P}_n\left(x\right)-\int n^c(x,\theta ,t,A,\tau )d{P}_0\left(x\right)\right|\to 0,$$

(69)

in probability. The following inequality holds

$$\begin{array}{c}\underset{t\in N_{t_{\theta }^c}}{sup}\left|\int n^c(x,\theta ,t,{\widehat{A}}_{\theta },{\widehat{\tau }}_{\theta })d{P}_n\left(x\right)-\int n^c(x,\theta ,t,A_{\theta },{\tau }_{\theta })d{P}_0\left(x\right)\right|\le \hfill \\ \underset{t\in N_{t_{\theta }^c}}{sup}\left|\int n^c(x,\theta ,t,{\widehat{A}}_{\theta },{\widehat{\tau }}_{\theta })d{P}_n\left(x\right)-\int n^c(x,\theta ,t,{\widehat{A}}_{\theta },{\widehat{\tau }}_{\theta })d{P}_0\left(x\right)\right|+\\ \hfill +\underset{t\in N_{t_{\theta }^c}}{sup}\left|\int n^c(x,\theta ,t,{\widehat{A}}_{\theta },{\widehat{\tau }}_{\theta })d{P}_0\left(x\right)-\int n^c(x,\theta ,t,A_{\theta },{\tau }_{\theta })d{P}_0\left(x\right)\right|.\end{array}$$

(70)

The first term in the right hand side of the Inequality (70), tends to 0 in probability, on the basis of the result (69). The second term in the right hand side of (70) also tends to 0 in probability. Indeed, using the convergence in probability $({\widehat{A}}_{\theta },{\widehat{\tau }}_{\theta })\to (A_{\theta },{\tau }_{\theta })$, by Assumption 2(b), we get the pointwise convergence for each t. Then, according to Corollary 2.1 from [28], using the point-wise convergence together with Assumption 2(c), we obtain the uniform convergence

$$\underset{t\in N_{t_{\theta }^c}}{sup}\left|\int n^c(x,\theta ,t,{\widehat{A}}_{\theta },{\widehat{\tau }}_{\theta })d{P}_0\left(x\right)-\int n^c(x,\theta ,t,A_{\theta },{\tau }_{\theta })d{P}_0\left(x\right)\right|=o_P\left(1\right).$$

(71)

Consequently

$$\underset{t\in N_{t_{\theta }^c}}{sup}\left|\int n^c(x,\theta ,t,{\widehat{A}}_{\theta },{\widehat{\tau }}_{\theta })d{P}_n\left(x\right)-\int n^c(x,\theta ,t,A_{\theta },{\tau }_{\theta })d{P}_0\left(x\right)\right|\to 0,$$

(72)

in probability. Using (72) and the fact that $t_{\theta }^c$ is unique and belongs to $\mathrm{int}\left(N_{t_{\theta }^c}\right)$ and the strict concavity of the function $t\mapsto \int n^c(y,\theta ,t,A_{\theta },{\tau }_{\theta })d{P}_0\left(y\right)$, on the basis of Theorem 5.7 in [27], we conclude that any value

$$\underline{t}:=\underset{t\in N_{t_{\theta }^c}}{argsup}\int n^c(x,\theta ,t,{\widehat{A}}_{\theta },{\widehat{\tau }}_{\theta })d{P}_n\left(x\right)$$

(73)

converges in probability to $t_{\theta }^c$. We show that $\widehat{t_{\theta }^c}$ belongs to $\mathrm{int}\left(N_{t_{\theta }^c}\right)$ with probability one as $n\to \infty$, and consequently it converges to $t_{\theta }^c$. Since for n sufficiently large any $\underline{t}\in \mathrm{int}\left(N_{t_{\theta }^c}\right)$, the concavity of the criterion function $t\mapsto \int n^c(x,\theta ,t,{\widehat{A}}_{\theta },{\widehat{\tau }}_{\theta })d{P}_n\left(x\right)$ ensures that no other point t in the complement of $\mathrm{int}\left(N_{t_{\theta }^c}\right)$ can maximize $\int n^c(x,\theta ,t,{\widehat{A}}_{\theta },{\widehat{\tau }}_{\theta })d{P}_n\left(x\right)$ over $t\in {\mathbb{R}}^{\ell }$; hence $\widehat{t_{\theta }^c}$ belongs to $\mathrm{int}\left(N_{t_{\theta }^c}\right)$.

(2) We have

$$\begin{array}{c}\widehat{K{L}_m}({\mathcal{M}}_{\theta }^{c,n},P_0)-K{L}_m({\mathcal{M}}_{\theta }^c,P_0)=\int n^c(x,\theta ,\widehat{t_{\theta }^c},{\widehat{A}}_{\theta },{\widehat{\tau }}_{\theta })d{P}_n\left(x\right)-\hfill \\ \hfill \int n^c(x,\theta ,t_{\theta }^c,A_{\theta },{\tau }_{\theta })d{P}_0\left(x\right).\end{array}$$

Note that

$$\begin{array}{c}\int n^c(x,\theta ,t_{\theta }^c,{\widehat{A}}_{\theta },{\widehat{\tau }}_{\theta })d{P}_n\left(x\right)-\int n^c(x,\theta ,t_{\theta }^c,A_{\theta },{\tau }_{\theta })d{P}_0\left(x\right)\le \hfill \\ \widehat{K{L}_m}({\mathcal{M}}_{\theta }^{c,n},P_0)-K{L}_m({\mathcal{M}}_{\theta }^{c,n},P_0)\le \\ \hfill \int n^c(x,\theta ,\widehat{t_{\theta }^c},{\widehat{A}}_{\theta },{\widehat{\tau }}_{\theta })d{P}_n\left(x\right)-\int n^c(x,\theta ,\widehat{t_{\theta }^c},A_{\theta },{\tau }_{\theta })d{P}_0\left(x\right).\end{array}$$

Both the right hand side and the left hand side in the above display tend to 0 in probability using (72). Hence, ${\widehat{KL}}_m({\mathcal{M}}_{\theta }^{c,n},P_0)$ converges to $K{L}_m({\mathcal{M}}_{\theta }^c,P_0)$, in probability. □

2.4.3. Consistency of the Estimator $\widehat{{\theta }^c}$

Assumption 3.$\left(\mathrm{a}\right)$ the function $(\theta ,t,A,\tau ))\mapsto h^c(X,\theta ,t,A,\tau )$ is continuous, with probability 1;

$\left(\mathrm{b}\right)$ for each $\theta \in \mathsf{\Theta }$, there exist compact neighborhood $N_{t_{\theta }^c}$ of $t_{\theta }^c$ and compact neighborhood $N_{\theta }$ of $(A_{\theta },{\tau }_{\theta })$ such that

$$\int \underset{\theta \in \mathsf{\Theta }}{sup}\underset{t\in N_{t_{\theta }^c},(A,\tau )\in N_{\theta }}{sup}\left|n^c(x,\theta ,t,A,\tau )\right|d{P}_0\left(x\right)<\infty ;$$

(74)

$\left(\mathrm{c}\right)$ Let $t_n^{*}\left(\theta \right):=\underset{t\in N_{t_{\theta }^c}}{argsup}\left|\int n^c(x,\theta ,t,{\widehat{A}}_{\theta },{\widehat{\tau }}_{\theta })d{P}_0\left(x\right)-\int n^c(x,\theta ,t,A_{\theta },{\tau }_{\theta })d{P}_0\left(x\right)\right|$. There exists a sequence $B_n=O_P\left(1\right)$ such that, for all $\theta ,{\theta }^{\prime }\in \mathsf{\Theta }$, it holds

$$\begin{array}{c}\left|\int n^c(x,\theta ,t_n^{*}\left(\theta \right),{\widehat{A}}_{\theta },{\widehat{\tau }}_{\theta })d{P}_0\left(x\right)-\right.\hfill \\ \hfill \left.-\int n^c(x,{\theta }^{\prime },t_n^{*}\left({\theta }^{\prime }\right),{\widehat{A}}_{{\theta }^{\prime }},{\widehat{\tau }}_{{\theta }^{\prime }})d{P}_0\left(x\right)\right|\le B_n\parallel \theta -{\theta }^{\prime }\parallel ;\end{array}$$

(75)

$\left(\mathrm{d}\right)$ the function $\theta \mapsto \int n^c(x,\theta ,t_{\theta }^c,A_{\theta },{\tau }_{\theta })d{P}_0\left(x\right)$ is continuous on Θ;

$\left(\mathrm{e}\right)$${\theta }_0:=\underset{\theta \in \mathsf{\Theta }}{arginf}\int n^c(x,\theta ,t_{\theta }^c,A_{\theta },{\tau }_{\theta })d{P}_0\left(x\right)$ exists, unique and interior point of Θ.

Proposition 4.

Under Assumptions 1–3, we have

$\left(1\right)$$\parallel \widehat{t_{\theta }^c}-t_{\theta }^c\parallel \to 0$ in probability, uniformly with respect to $\theta \in \mathsf{\Theta }$;

$\left(2\right)$$\parallel \widehat{{\theta }^c}-{\theta }_0\parallel \to 0$ in probability.

Proof. 

(1) By Assumption 3(a), $n^c(x,\theta ,t,A,\tau )$ is continuous in $\theta ,t,A,\tau$. Using also Assumption 3(b), by applying UWLLN, we obtain the uniform convergence in probability

$$\underset{(\theta ,t,A,\tau )\in C}{sup}\left|\int n^c(x,\theta ,t,A,\tau )d{P}_n\left(x\right)-\int n^c(x,\theta ,t,A,\tau )d{P}_0\left(x\right)\right|\to 0$$

(76)

over the compact set $C:=\left\{(\theta ,t,A,\tau )\mathrm{s}.\mathrm{t}.\theta \in \mathsf{\Theta },t\in N_{t_{\theta }^c},(A,\tau )\in N_{\theta }\right\}.$ We will prove the uniform convergence in probability

$$\underset{\theta \in \mathsf{\Theta }}{sup}\parallel \widehat{t_{\theta }^c}-t_{\theta }^c\parallel \to 0.$$

(77)

Let $\eta >0$. We show that $P_0({sup}_{\theta \in \mathsf{\Theta }}\parallel \widetilde{t_{\theta }^c}-t_{\theta }^c\parallel \ge \eta )\to 0$ for any

$$\widetilde{t_{\theta }^c}:=\underset{t\in N_{t_{\theta }^c}}{argsup}\int n^c(x,\theta ,t,{\widehat{A}}_{\theta },{\widehat{\tau }}_{\theta })d{P}_n\left(x\right).$$

(78)

We show that $\widehat{t_{\theta }^c}$ belongs to $\mathrm{int}\left(N_{t_{\theta }^c}\right)$ with probability one, as $n\to \infty .$

Let $\eta >0$ be such that ${sup}_{\theta \in \mathsf{\Theta }}\parallel \widetilde{t_{\theta }^c}-t_{\theta }^c\parallel \ge \eta$. Since $\mathsf{\Theta }$ is compact, by continuity, there exists $\overline{\theta }\in \mathsf{\Theta }$, such that ${sup}_{\theta \in \mathsf{\Theta }}\parallel \widetilde{t_{\theta }^c}-t_{\theta }^c\parallel =\parallel \widetilde{t_{\overline{\theta }}^c}-t_{\overline{\theta }}^c\parallel \ge \eta .$ Hence, there exists $\epsilon >0$ such that

$$\begin{array}{c}\hfill \int n^c(x,\overline{\theta },t_{\overline{\theta }}^c,A_{\overline{\theta }},{\tau }_{\overline{\theta }})d{P}_0\left(x\right)-\int n^c(x,\overline{\theta },{\tilde{t}}_{\overline{\theta }}^c,A_{\overline{\theta }},{\tau }_{\overline{\theta }})d{P}_0\left(x\right)>\epsilon .\end{array}$$

Therefore,

$$\begin{array}{c}{P}_0\left(\underset{\theta \in \mathsf{\Theta }}{sup}\parallel \widetilde{t_{\theta }^c}-t_{\theta }^c\parallel \ge \eta \right)\le P_0\left(\int n^c(x,\overline{\theta },t_{\overline{\theta }}^c,A_{\overline{\theta }},{\tau }_{\overline{\theta }})d{P}_0\left(x\right)-\right.\hfill \\ \hfill \left.\int n^c(x,\overline{\theta },\widetilde{t_{\overline{\theta }}^c},A_{\overline{\theta }},{\tau }_{\overline{\theta }})d{P}_0\left(x\right)>\epsilon \right).\end{array}$$

(79)

On the other hand, using (71) and (72), we can write

$$\begin{array}{c}\int n^c(x,\overline{\theta },t_{\overline{\theta }}^c,A_{\overline{\theta }},{\tau }_{\overline{\theta }})d{P}_0\left(x\right)-\int n^c(x,\overline{\theta },{\tilde{t}}_{\overline{\theta }}^c,A_{\overline{\theta }},{\tau }_{\overline{\theta }})d{P}_0\left(x\right)=\hfill \\ \int n^c(x,\overline{\theta },t_{\overline{\theta }}^c,{\widehat{A}}_{\overline{\theta }},{\widehat{\tau }}_{\overline{\theta }})d{P}_n\left(x\right)-\int n^c(x,\overline{\theta },{\tilde{t}}_{\overline{\theta }}^c,A_{\overline{\theta }},{\tau }_{\overline{\theta }})d{P}_0\left(x\right)+o_P\left(1\right)\le \\ \le \int n^c(x,\overline{\theta },{\tilde{t}}_{\overline{\theta }}^c,{\widehat{A}}_{\overline{\theta }},{\widehat{\tau }}_{\overline{\theta }})d{P}_n\left(x\right)-\int n^c(x,\overline{\theta },{\tilde{t}}_{\overline{\theta }}^c,{\widehat{A}}_{\overline{\theta }},{\widehat{\tau }}_{\overline{\theta }})d{P}_0\left(x\right)+o_P\left(1\right)\le \\ \hfill \le {\displaystyle \underset{C}{sup}}\left|\int n^c(x,\theta ,t,A,\tau )d{P}_n\left(x\right)-\int n^c(x,\theta ,t,A,\tau )d{P}_0\left(x\right)\right|+o_P\left(1\right).\end{array}$$

(80)

Using (76), (79), and (80), we deduce that ${sup}_{\theta \in \mathsf{\Theta }}\parallel \widetilde{t_{\theta }^c}-t_{\theta }^c\parallel \to 0$ in probability. In particular, for large n, $\widetilde{t_{\theta }^c}\in \mathrm{int}\left(N_{t_{\theta }^c}\right)$ uniformly in $\theta$. Since $t\mapsto \int n^c(x,\theta ,t,{\widehat{A}}_{\theta },{\widehat{\tau }}_{\theta })d{P}_n\left(x\right)$ is concave, the maximizer $\widehat{t_{\theta }^c}$ belongs to $\mathrm{int}\left(N_{t_{\theta }^c}\right)$ for sufficiently large n. Then, ${sup}_{\theta \in \mathsf{\Theta }}\parallel \widehat{t_{\theta }^c}-t_{\theta }^c\parallel \to 0$ in probability. (2) For large n, we can write

$$\begin{array}{c}\underset{\theta \in \mathsf{\Theta }}{sup}\left|\int n^c(x,\theta ,\widehat{t_{\theta }^c},{\widehat{A}}_{\theta },{\widehat{\tau }}_{\theta })d{P}_n\left(x\right)-\int n^c(x,\theta ,t_{\theta }^c,A_{\theta },{\tau }_{\theta })d{P}_0\left(x\right)\right|\hfill \\ \hfill =\underset{\theta \in \mathsf{\Theta }}{sup}\left|\int n^c(x,\theta ,\widetilde{t_{\theta }^c},{\widehat{A}}_{\theta },{\widehat{\tau }}_{\theta })d{P}_n\left(x\right)-\int n^c(x,\theta ,t_{\theta }^c,A_{\theta },{\tau }_{\theta })d{P}_0\left(x\right)\right|=:\underset{\theta \in \mathsf{\Theta }}{sup}\left|B\right|.\end{array}$$

(81)

Note that

$$\begin{array}{c}\int n^c(x,\theta ,t_{\theta }^c,{\widehat{A}}_{\theta },{\widehat{\tau }}_{\theta })d{P}_n\left(x\right)-\int n^c(x,\theta ,t_{\theta }^c,A_{\theta },{\tau }_{\theta })d{P}_0\left(x\right)\le \hfill \\ \hfill \le B\le \int n^c(x,\theta ,\widetilde{t_{\theta }^c},{\widehat{A}}_{\theta },{\widehat{\tau }}_{\theta })d{P}_n\left(x\right)-\int n^c(x,\theta ,\widetilde{t_{\theta }^c},A_{\theta },{\tau }_{\theta })d{P}_0\left(x\right).\end{array}$$

(82)

In order to prove that ${sup}_{\theta \in \mathsf{\Theta }}\left|B\right|\to 0$ in probability, we first prove that

$$\underset{\theta \in \mathsf{\Theta }}{sup}\underset{t\in N_{t_{\theta }^c}}{sup}\left|\int n^c(x,\theta ,t,{\widehat{A}}_{\theta },{\widehat{\tau }}_{\theta })d{P}_n\left(x\right)-\int n^c(x,\theta ,t,A_{\theta },{\tau }_{\theta })d{P}_0\left(x\right)\right|\to 0,$$

(83)

in probability. Notice that

$$\begin{array}{c}\underset{\theta \in \mathsf{\Theta }}{sup}\underset{t\in N_{t_{\theta }^c}}{sup}\left|\int n^c(x,\theta ,t,{\widehat{A}}_{\theta },{\widehat{\tau }}_{\theta })d{P}_n\left(x\right)-\int n^c(x,\theta ,t,A_{\theta },{\tau }_{\theta })d{P}_0\left(x\right)\right|\le \hfill \\ \underset{\theta \in \mathsf{\Theta }}{sup}\underset{t\in N_{t_{\theta }^c}}{sup}\left|\int n^c(x,\theta ,t,{\widehat{A}}_{\theta },{\widehat{\tau }}_{\theta })d{P}_n\left(x\right)-\int n^c(x,\theta ,t,{\widehat{A}}_{\theta },{\widehat{\tau }}_{\theta })d{P}_0\left(x\right)\right|+\\ \hfill \underset{\theta \in \mathsf{\Theta }}{sup}\underset{t\in N_{t_{\theta }^c}}{sup}\left|\int n^c(x,\theta ,t,{\widehat{A}}_{\theta },{\widehat{\tau }}_{\theta })d{P}_0\left(x\right)-\int n^c(x,\theta ,t,A_{\theta },{\tau }_{\theta })d{P}_0\left(x\right)\right|.\end{array}$$

The first term in the right-hand side of the above inequality tends to 0 in probability, using (76). Regarding the second term, we have the convergence (71), and combining this with Assumption 3(c), we obtain

$$\underset{\theta \in \mathsf{\Theta }}{sup}\underset{t\in N_{t_{\theta }^c}}{sup}\left|\int n^c(x,\theta ,t,{\widehat{A}}_{\theta },{\widehat{\tau }}_{\theta })d{P}_0\left(x\right)-\int n^c(x,\theta ,t,A_{\theta },{\tau }_{\theta })d{P}_0\left(x\right)\right|\to 0,$$

(84)

in probability. Consequently, (83) holds. Then, (81) and (82) lead to

$$\underset{\theta \in \mathsf{\Theta }}{sup}\left|\int n^c(x,\theta ,\widehat{t_{\theta }^c},{\widehat{A}}_{\theta },{\widehat{\tau }}_{\theta })d{P}_n\left(x\right)-\int n^c(x,\theta ,t_{\theta }^c,A_{\theta },{\tau }_{\theta })d{P}_0\left(x\right)\right|\to 0.$$

(85)

Assumptions 3(d) and (e) ensure that ${\theta }_0$ is well-separated in the sense that, $\forall \epsilon >0$,

$$\begin{array}{c}\hfill \underset{\{\theta ;\parallel \theta -{\theta }_0\parallel \ge \epsilon \}}{inf}\int n^c(x,\theta ,t_{\theta }^c,A_{\theta },{\tau }_{\theta })d{P}_0\left(x\right)>0=\int n^c(x,{\theta }_0,t_{\theta }^c,A_{{\theta }_0},{\tau }_{{\theta }_0})d{P}_0\left(x\right).\end{array}$$

(86)

Finally, (85) and (86) imply that $\widehat{{\theta }^c}\to {\theta }_0$ in probability, on the basis of Theorem 5.7 p. 45 from [27]. □

3. Simulation Results

In order to compare the performance of the proposed robust EL estimate (26) with that of the EL estimator (27) in the case of contaminated data, we consider the moment condition model presented in Example 1 in Section 1.

Let X be a random variable with probability distribution $P_0={\chi }_1^2$, a chi-square distribution with one degree of freedom. Then, it holds that the equation ${\int }_{\mathbb{R}}g(x,\theta )d{P}_0\left(x\right)=0$, with $g(x,\theta )={(x-\theta ,x^2-{\theta }^2-2\theta )}^{\top }$, has a unique solution $\theta ={\theta }_0=1$. This is a particular case of the model from Example 1, namely when $h\left(\theta \right)={\theta }^2+\theta .$ Observe that, for this model, $g(x,\theta )$ is unbounded (in x).

We use i.i.d. data generated from a slight deviation of the model $P_0$, namely from the model

$$(1-ϵ){\chi }_1^2+ϵ{\chi }_{10}^2,$$

with $ϵ=0.05$, respectively $ϵ=0.10$. The considered sample sizes are $n=100$ and $n=500.$ All the simulations are repeated 1000 times. The obtained estimates are compared through bias, variance, and mean square error, computed on the basis of the 1000 replications. We give the corresponding box-plots in Figure 1, Figure 2, Figure 3 and Figure 4, where the true value of the parameter ${\theta }_0=1$ is presented with horizontal dashed line. For computing the proposed Robust EL estimate (26), we use the truncated functions $g^c(x,\theta )$ and $g_n^c(x,\theta )$ with $c=2$. The algorithm for computing the estimate was obtained by adapting the one from [10] (Appendix A.1., p. 3196). Namely, for each iteration, the estimation of the parameter ${\theta }_0$, corresponding to the new orthogonality function in step iii, is computed using Uzawa algorithm for the saddle-point optimum in (26). The obtained results are presented in

Table 1. Robust Empirical Likelihood (EL) versus EL, for $n=100$ and $ϵ=0.05$.

	Estimate	Squared Bias	Variance	Mean Square Error
Robust EL	$1.0477$	$0.0023$	$0.0034$	$0.0056$
EL	$1.0861$	$0.0074$	$0.0208$	$0.0282$

,

Table 2. Robust EL versus EL, for $n=100$ and $ϵ=0.10$.

	Estimate	Squared Bias	Variance	Mean Square Error
Robust EL	$1.1281$	$0.0164$	$0.0116$	$0.0281$
EL	$1.2052$	$0.0421$	$0.0342$	$0.0763$

,

Table 3. Robust EL versus EL, for $n=500$ and $ϵ=0.05$.

	Estimate	Squared Bias	Variance	Mean Square Error
Robust EL	$1.0336$	$0.0011$	$0.0005$	$0.0016$
EL	$1.0750$	$0.0056$	$0.0039$	$0.0095$

and

Table 4. Robust EL versus EL, for $n=500$ and $ϵ=0.10$.

	Estimate	Squared Bias	Variance	Mean Square Error
Robust EL	$1.1033$	$0.0107$	$0.0019$	$0.0126$
EL	$1.1835$	$0.0337$	$0.0065$	$0.0402$

and Figure 1, Figure 2, Figure 3 and Figure 4. All these results illustrate the fact that, in the case of contaminated data, the robust EL estimator outperforms the classical EL estimator.

4. Conclusions

We proposed a robust version of the EL estimator for moment condition models. This estimator is defined through the minimization of an empirical version of the modified Kullback–Leibler divergence in dual form, using truncated orthogonality functions based on multivariate Huber function. We proved the robustness by means of the influence function, as well the consistency of the new estimator. The results of the Monte Carlo simulation study show that, in the case of contaminated data, the robust EL estimator outperforms the classical EL estimator.