Robust Z-Estimators for Semiparametric Moment Condition Models

Abstract

In the present paper, we introduce a class of robust Z-estimators for moment condition models. These new estimators can be seen as robust alternatives for the minimum empirical divergence estimators. By using the multidimensional Huber function, we first define robust estimators of the element that realizes the supremum in the dual form of the divergence. A linear relationship between the influence function of a minimum empirical divergence estimator and the influence function of the estimator of the element that realizes the supremum in the dual form of the divergence led to the idea of defining new Z-estimators for the parameter of the model, by using robust estimators in the dual form of the divergence. The asymptotic properties of the proposed estimators were proven, including here the consistency and their asymptotic normality. Then, the influence functions of the estimators were derived, and their robustness is demonstrated.

1. Introduction

A moment condition model is a family ${\mathcal{M}}^1$ of probability measures, all defined on the same measurable space $({\mathbb{R}}^m,\mathcal{B}\left({\mathbb{R}}^m\right))$, such that

$$\int g(x,\theta )dQ\left(x\right)=0,\mathrm{for}\mathrm{all}Q\in {\mathcal{M}}^1.$$

(1)

The parameter $\theta$ belongs to $\mathsf{\Theta }\subset {\mathbb{R}}^d$; the function $g:={(g_1,\dots ,g_l)}^{\top }$ is defined on ${\mathbb{R}}^m\times \mathsf{\Theta }$, each of the $g_i$’s being real-valued, $l\ge d$, and the functions $g_1,\dots ,g_l$ and ${\mathbf{1}}_{\mathcal{X}}$ are supposed to be linearly independent. Denote by $M^1$ the set of all probability measures on $({\mathbb{R}}^m,\mathcal{B}\left({\mathbb{R}}^m\right))$ and

$${\mathcal{M}}_{\theta }^1:=\{Q\in M^1:\int g(x,\theta )dQ\left(x\right)=0\},$$

(2)

such that

$${\mathcal{M}}^1=\bigcup _{\theta \in \mathsf{\Theta }}{\mathcal{M}}_{\theta }^1.$$

(3)

Let $X_1,\cdots ,X_n$ be an i.i.d. sample on the random vector X with unknown probability distribution $P_0$. We considered the problem of the estimation of the parameter ${\theta }_0$ for which the constraints of the model are satisfied:

$$\int g(x,{\theta }_0)d{P}_0\left(x\right)=0.$$

(4)

We supposed that ${\theta }_0$ is the unique solution of Equation (4). Thus, we assumed that information about ${\theta }_0$ and $P_0$ is available in the form of $l\ge d$ functionally independent unbiased estimating functions, and we used this information to estimate ${\theta }_0$.

Among the best-known estimation methods for moment condition models, we mention the generalized method of moments (GMM) [1], the continuous updating (CU) estimator [2], the empirical likelihood (EL) estimator [3,4], the exponential tilting (ET) estimator [5], and the generalized empirical likelihood (GEL) estimators [6]. Although the EL estimator is superior to other estimators in terms of higher-order asymptotic properties, these properties hold only under the correct specification of the moment conditions. In [7] was proposed the exponentially tilted empirical likelihood (ETEL) estimator, which has the same higher-order property as the EL estimator under the correct specification, while maintaining the usual asymptotic properties such as the consistency and asymptotic normality under misspecification. The so-called information and entropy econometric techniques have been proposed to improve the finite sample performance of the GMM-estimators and tests (see, e.g., [4,5]).

Some recent methods for the estimation and testing of moment condition models are based on using divergences. Divergences between probability measures are widely used in statistics and data science in order to perform inference in models of various kinds, parametric or semiparametric. Statistical methods based on divergence minimization extend the likelihood paradigm and often have the advantage of providing a trade-off between efficiency and robustness [8,9,10,11]. A general methodology for the estimation and testing of moment condition models was developed in [12]. This approach is based on minimizing divergences in their dual form and allows the asymptotic study of the estimators, called minimum empirical divergence estimators, and of the associated test statistics, both under the model and under misspecification of the model. The approach based on minimizing dual forms of divergences was initially used in the case of parametric models, the results being published in a series of articles [13,14,15,16]. The broad class of minimum empirical divergence estimators contains in particular the EL estimator, the CU estimator, as well as the ET estimator mentioned above. Using the influence function as the robustness measure, it has been shown that the minimum empirical divergence estimators are not robust, because the corresponding influence functions are generally not bounded [17]. On the other hand, the minimum empirical divergence estimators have the same efficiency of first order, and moreover, the EL estimator, which belong to this class, is superior in higher-order efficiency. Therefore, proposing robust versions of the minimum empirical divergence estimators would bring a trade-off between robustness and efficiency. These aspects motivated our studies in the present paper.

Some robust estimation methods for moment condition models have been proposed in the literature, for example in [18,19,20,21,22]. In the present paper, we introduce a class of robust Z-estimators for moment condition models. These new estimators can be seen as robust alternatives for the minimum empirical divergence estimators. By using the multidimensional Huber function, we first define robust estimators of the element that realizes the supremum in the dual form of the divergence. A linear relationship between the influence function of a minimum empirical divergence estimator and the influence function of the estimator of the element that realizes the supremum in the dual form of the divergence led to the idea of defining new Z-estimators for the parameter of the model, by using robust estimators in the dual form of the divergence. The asymptotic properties of the proposed estimators were proven, including here the consistency and their asymptotic normality. Then, the influence functions of the estimators were derived, and their robustness is demonstrated.

The paper is organized as follows. In Section 2, we briefly recall the context and the definitions of the minimum empirical divergence estimators, these being necessary for defining the new estimators. In Section 3, the new Z-estimators for moment condition models are defined. The asymptotic properties of these estimators were proven, including here the consistency and their asymptotic normality. Then, the influence functions of the estimators were derived, and their robustness is demonstrated. The proofs of the theoretical results are deferred in Appendix A.

2. Minimum Empirical Divergence Estimators

2.1. Statistical Divergences

Let $\phi$ be a convex function defined on $\mathbb{R}$ and $\left[0,\infty \right]$-valued, such that $\phi \left(1\right)=0$, and let $P\in M^1$ be some probability measure. For any signed finite measure Q defined on the same measurable space $({\mathbb{R}}^m,\mathcal{B}\left({\mathbb{R}}^m\right))$, absolutely continuous (a.c.) with respect to P, the $\phi$ divergence between Q and P is defined by

$$D_{\phi }(Q,P):=\int \phi \left(\frac{dQ}{dP}\left(x\right)\right)dP\left(x\right).$$

(5)

When Q is not a.c. with respect to $P,$ we set $D_{\phi }(Q,P)=\infty$. This extension, for the case when Q is not absolutely continuous with respect to P, was considered in order to have a unique definition of divergences, appropriate for both cases—that of continuous probability laws and that of discrete probability laws.

This definition extends the one of divergences between probability measures [23], and the necessity of working with signed finite measures will be explained in Section 2.2.

Largely used in information theory, the Kullback–Leibler divergence is associated with the real convex function $\phi \left(x\right):=xlogx-x+1$ and is defined by

$$KL(Q,P):=\int log\left(\frac{dQ}{dP}\right)dQ.$$

The modified Kullback–Leibler divergence is associated with the convex function $\phi \left(x\right):=-logx+x-1$ and is defined through

$$K{L}_m(Q,P):=\int -log\left(\frac{dQ}{dP}\right)dP.$$

Other divergences, largely used in inferential statistics, are the ${\chi }^2$ and the modified ${\chi }^2$ divergences, namely

$${\chi }^2(Q,P):=\frac{1}{2}\int {\left(\frac{dQ}{dP}-1\right)}^{2}dP,$$

$${\chi }_m^2(Q,P):=\frac{1}{2}\int {\displaystyle \frac{{\left(\frac{dQ}{dP}-1\right)}^2}{\frac{dQ}{dP}}}dP,$$

these being associated with the convex functions $\phi \left(x\right):=\frac{1}{2}{(x-1)}^2$ and $\phi \left(x\right):=\frac{1}{2}{(x-1)}^2/x$, respectively. The Hellinger distance and the $L_1$ distance are also $\phi$ divergences. They are associated with the convex functions $\phi \left(x\right):=2{(\sqrt{x}-1)}^2$ and $\phi \left(x\right):=|x-1|$, respectively.

All the preceding examples, except the $L_1$ distance, belong to the class of power divergences introduced by Cressie and Read [24] and defined by the convex functions:

$$x\in {\mathbb{R}}_{+}^{*}\mapsto {\phi }_{\gamma }\left(x\right):=\frac{x^{\gamma }-\gamma x+\gamma -1}{\gamma (\gamma -1)},$$

(6)

for $\gamma \in \mathbb{R}\setminus \{0,1\}$ and ${\phi }_0\left(x\right):=-logx+x-1$, ${\phi }_1\left(x\right):=xlogx-x+1$. The Kullback–Leibler divergence is associated with ${\phi }_1$, the modified Kullback–Leibler with ${\phi }_0$, the ${\chi }^2$ divergence with ${\phi }_2$, the modified ${\chi }^2$ divergence with ${\phi }_{-1}$, and the Hellinger distance with ${\phi }_{1/2}$. When ${\phi }_{\gamma }$ is not defined on $(-\infty ,0)$ or when ${\phi }_{\gamma }$ is not convex, the definition of the corresponding power divergence function $Q\in M^1\mapsto D_{{\phi }_{\gamma }}(Q,P)$ can be extended to the whole set of signed finite measures by taking the following extension of ${\phi }_{\gamma }$:

$${\phi }_{\gamma }:x\in \mathbb{R}\mapsto {\phi }_{\gamma }\left(x\right){\mathbf{1}}_{\left[0,\infty \right)}\left(x\right)+(+\infty ){\mathbf{1}}_{\left(-\infty ,0\right)}\left(x\right).$$

The $\phi$ divergence between some set $\mathsf{\Omega }$ of signed finite measures and a probability measure P is defined by

$$D_{\phi }(\mathsf{\Omega },P)=\underset{Q\in \mathsf{\Omega }}{inf}{D}_{\phi }(Q,P).$$

(7)

Assuming that $D_{\phi }(\mathsf{\Omega },P)$ is finite, a measure $Q^{*}\in \mathsf{\Omega }$ is called a $\phi$-projection of P on $\mathsf{\Omega }$ if

$$D_{\phi }(Q^{*},P)\le D_{\phi }(Q,P),\mathrm{for}\mathrm{all}Q\in \mathsf{\Omega }.$$

2.2. Minimum Empirical Divergence Estimators

Let $X_1,\cdots ,X_n$ be an i.i.d. sample on the random vector X with the probability distribution $P_0$. The “plug-in” estimator of the $\phi$ divergence $D_{\phi }({\mathcal{M}}_{\theta }^1,P_0)$ between the set ${\mathcal{M}}_{\theta }^1$ and the probability measure $P_0$ is defined by replacing $P_0$ with the empirical measure associated with the sample. More precisely,

$${\widehat{D}}_{\phi }({\mathcal{M}}_{\theta }^1,P_0)=\underset{Q\in {\mathcal{M}}_{\theta }^1}{inf}{D}_{\phi }(Q,P_n)=\underset{Q\in {\mathcal{M}}_{\theta }^1}{inf}\int \phi \left(\frac{dQ}{d{P}_n}\left(x\right)\right)d{P}_n\left(x\right),$$

(8)

where $P_n:=\frac{1}{n}{\sum }_{i=1}^n{\delta }_{X_i}$ is the empirical measure associated with the sample, ${\delta }_x$ being the Dirac measure putting all mass at x. If the projection of the measure $P_n$ on ${\mathcal{M}}_{\theta }^1$ exists, it is a law a.c. with respect to $P_n$. Then, it is natural to consider

$${\mathcal{M}}_{\theta }^{\left(n\right)}=\{Q\in M^1:Q\mathrm{a}.\mathrm{c}.\mathrm{with}\mathrm{respect}\mathrm{to}{P}_n\mathrm{and}\sum _{i=1}^{n}g(X_i,\theta )Q\left(X_i\right)=0\},$$

(9)

and then, the plug-in estimator (8) can be written as

$${\widehat{D}}_{\phi }({\mathcal{M}}_{\theta }^1,P_0)=\underset{Q\in {\mathcal{M}}_{\theta }^{\left(n\right)}}{inf}\frac{1}{n}\sum _{i=1}^n\phi \left(nQ\left(X_i\right)\right).$$

(10)

The infimum in the above expression (10) may be achieved at a point situated on the frontier of the set ${\mathcal{M}}_{\theta }^{\left(n\right)}$, a case in which the Lagrange method for characterizing the infimum and computing ${\widehat{D}}_{\phi }({\mathcal{M}}_{\theta }^1,P_0)$ cannot be applied. In order to avoid this difficulty, Broniatowski and Keziou [12,25] proposed to work on sets of signed finite measures and defined

$${\mathcal{M}}_{\theta }:=\{Q\in M:\int dQ=1\mathrm{and}\int g(x,\theta )dQ\left(x\right)=0\},$$

(11)

where M denotes the set of all signed finite measures on the measurable space $({\mathbb{R}}^m,\mathcal{B}\left({\mathbb{R}}^m\right))$, and

$$\mathcal{M}:=\bigcup _{\theta \in \mathsf{\Theta }}{\mathcal{M}}_{\theta }.$$

(12)

They showed that, if $Q_1^{*}$ the projection of $P_n$ on ${\mathcal{M}}_{\theta }^1$ is an interior point of ${\mathcal{M}}_{\theta }^1$ and $Q^{*}$ the projection of $P_n$ on ${\mathcal{M}}_{\theta }$ is an interior point of ${\mathcal{M}}_{\theta }$, then both approaches based on signed finite measures, respectively on probability measures, for defining minimum divergence estimators coincide. On the other hand, in the case when $Q_1^{*}$ is a frontier point of ${\mathcal{M}}_{\theta }^1$, the estimator of the parameter ${\theta }_0$ defined using the context of signed finite measures converges to ${\theta }_0$. These aspects justify the substitution of ${\mathcal{M}}_{\theta }^1$ by ${\mathcal{M}}_{\theta }$.

In the following, we briefly recall the definitions of the estimators for the moment condition proposed in [12] in the context of signed finite measure sets.

Denote by $\overline{g}$ the function defined on ${\mathbb{R}}^m\times \mathsf{\Theta }$ and ${\mathbb{R}}^{l+1}$-valued:

$$\overline{g}(x,\theta ):={\left({\mathbf{1}}_{\mathcal{X}}\left(x\right),g_1(x,\theta ),\dots ,g_l(x,\theta )\right)}^{\top }.$$

(13)

Given a $\phi$ divergence, when the function $\phi$ is strictly convex on its domain, denote

$${\phi }^{*}\left(u\right):=u{\phi \prime }^{-1}\left(u\right)-\phi \left({\phi \prime }^{-1}\left(u\right)\right),$$

the convex conjugate of the function $\phi$. For a given probability measure $P\in M^1$ and a fixed $\theta \in \mathsf{\Theta }$, define

$${\mathsf{\Lambda }}_{\theta }\left(P\right):=\{t\in {\mathbb{R}}^{l+1}:\int |{\phi }^{*}(t_0+\sum _{j=1}^l{t}_j{g}_j(x,\theta ))|dP\left(x\right)<\infty \}.$$

(14)

We also use the notations ${\mathsf{\Lambda }}_{\theta }$ for ${\mathsf{\Lambda }}_{\theta }\left(P_0\right)$ and ${\mathsf{\Lambda }}_{\theta }^{\left(n\right)}$ for ${\mathsf{\Lambda }}_{\theta }\left(P_n\right)$.

Supposing that $P_0$ admits a projection $Q_{\theta }^{*}$ on ${\mathcal{M}}_{\theta }$ with the same support as $P_0$ and that the function $\phi$ is strictly convex on its domain, then the $\phi$ divergence $D_{\phi }({\mathcal{M}}_{\theta },P_0)$ admits the dual representation:

$$D_{\phi }({\mathcal{M}}_{\theta },P_0)=\underset{t\in {\mathsf{\Lambda }}_{\theta }}{sup}\int m(x,\theta ,t)d{P}_0\left(x\right),$$

(15)

where $m(x,\theta ,t):=t_0-{\phi }^{*}\left(t^{\top }\overline{g}(x,\theta )\right).$

The supremum in (15) is unique and is reached at a point that we denote as $t_{\theta }=t_{\theta }\left(P_0\right)$:

$$t_{\theta }:=arg\underset{t\in {\mathsf{\Lambda }}_{\theta }}{sup}\int m(x,\theta ,t)d{P}_0\left(x\right).$$

(16)

Then, $D_{\phi }({\mathcal{M}}_{\theta },P_0)$, $t_{\theta }$, $D_{\phi }(\mathcal{M},P_0)$ and ${\theta }_0$ can be estimated respectively by

$$\begin{array}{ccc}& & {\widehat{D}}_{\phi }({\mathcal{M}}_{\theta },P_0):=\underset{t\in {\mathsf{\Lambda }}_{\theta }^{\left(n\right)}}{sup}\int m(x,\theta ,t)d{P}_n\left(x\right),\hfill \end{array}$$

(17)

$$\begin{array}{c}{\widehat{t}}_{\theta }:=arg\underset{t\in {\mathsf{\Lambda }}_{\theta }^{\left(n\right)}}{sup}\int m(x,\theta ,t)d{P}_n\left(x\right),\hfill \end{array}$$

(18)

$$\begin{array}{ccc}& & {\widehat{D}}_{\phi }(\mathcal{M},P_0):=\underset{\theta \in \mathsf{\Theta }}{inf}\underset{t\in {\mathsf{\Lambda }}_{\theta }^{\left(n\right)}}{sup}\int m(x,\theta ,t)d{P}_n\left(x\right),\hfill \end{array}$$

(19)

$$\begin{array}{ccc}& & {\widehat{\theta }}_{\phi }:=arg\underset{\theta \in \mathsf{\Theta }}{inf}\underset{t\in {\mathsf{\Lambda }}_{\theta }^{\left(n\right)}}{sup}\int m(x,\theta ,t)d{P}_n\left(x\right).\hfill \end{array}$$

(20)

The estimators defined in (20) are called minimum empirical divergence estimators. We refer to [12] for the complete study of the existence and of the asymptotic properties of the above estimators.

The influence functions of these estimators and corresponding robustness properties were studied in [17]. According to those results, for $\theta \in \mathsf{\Theta }$ fixed, the influence function of the estimator ${\widehat{t}}_{\theta }$ is given by

$$\mathrm{IF}(x;t_{\theta },P_0)=-{\left[\int \frac{{\partial }^2}{{\partial }^{2}t}m(y,\theta ,t_{\theta }\left(P_0\right))d{P}_0\left(y\right)\right]}^{-1}\frac{\partial }{\partial t}m(x,\theta ,t_{\theta }\left(P_0\right)),$$

(21)

where

$$\frac{\partial }{\partial t}m(x,\theta ,t)={(1,0_l)}^{\top }-{\phi \prime }^{-1}\left(t^{\top }\overline{g}(x,\theta )\right)\overline{g}(x,\theta ),$$

(22)

$$\frac{{\partial }^2}{{\partial }^{2}t}m(x,\theta ,t)=-\frac{1}{\phi ″\left({\phi \prime }^{-1}\left(t^{\top }\overline{g}(x,\theta )\right)\right)}\overline{g}(x,\theta )\overline{g}{(x,\theta )}^{\top },$$

(23)

with the particular case $\theta ={\theta }_0$:

$$\mathrm{IF}(x;t_{{\theta }_0},P_0)=-\phi ″\left(1\right){\left[\int \overline{g}(y,{\theta }_0)\overline{g}{(y,{\theta }_0)}^{\top }d{P}_0\left(y\right)\right]}^{-1}{(0,g{(x,{\theta }_0)}^{\top })}^{\top }.$$

(24)

On the other hand, the influence function of the estimator ${\widehat{\theta }}_{\phi }$ is given by

$$\begin{array}{c}\mathrm{IF}(x;T_{\phi },P_0)=-{\left\{{\left[\int \frac{\partial }{\partial \theta }g(y,{\theta }_0)d{P}_0\left(y\right)\right]}^{\top }{\left[\int g(y,{\theta }_0)g{(y,{\theta }_0)}^{\top }d{P}_0\left(y\right)\right]}^{-1}\int \frac{\partial }{\partial \theta }g(y,{\theta }_0)d{P}_0\left(y\right)\right\}}^{-1}\\ ·{\left[\int \frac{\partial }{\partial \theta }g(y,{\theta }_0)d{P}_0\left(y\right)\right]}^{\top }{\left[\int g(y,{\theta }_0)g{(y,{\theta }_0)}^{\top }d{P}_0\left(y\right)\right]}^{-1}g(x,{\theta }_0).\end{array}$$

(25)

Since the function $x\mapsto g(x,\theta )$ is usually not bounded, for example, when we have linear constraints, the influence function $\mathrm{IF}(x;T_{\phi },P_0)$ is not bounded; therefore, the minimum empirical divergence estimators ${\widehat{\theta }}_{\phi }$ defined in (20) are generally not robust.

Through the calculations, it can be seen that there is a connection between the influence functions $\mathrm{IF}(x;t_{{\theta }_0},P_0)$ and $\mathrm{IF}(x;T_{\phi },P_0)$, namely the relation

$${\left[\int \frac{\partial }{\partial \theta }\overline{g}(y,{\theta }_0)d{P}_0\left(y\right)\right]}^{\top }·\left\{\frac{\partial }{\partial \theta }t({\theta }_0,P_0)\mathrm{IF}(x;T_{\phi },P_0)+\mathrm{IF}(x;t_{{\theta }_0},P_0)\right\}=0.$$

Since $\mathrm{IF}(x;T_{\phi },P_0)$ is linearly related to $\mathrm{IF}(x;t_{{\theta }_0},P_0)$, using a robust estimator of $t_{\theta }=t_{\theta }\left(P_0\right)$ in the original duality Formula (15) would lead to a new robust estimator of ${\theta }_0$. This is the idea at the basis of our proposal in this paper, for constructing new robust estimators for moment condition models.

3. Robust Estimators for Moment Condition Models

3.1. Definitions of New Estimators

In this section, we define robust versions of the estimators ${\widehat{t}}_{\theta }$ from (18) and robust versions of minimum empirical divergence estimators ${\widehat{\theta }}_{\phi }$ from (20). First, we define robust estimators of $t_{\theta }$, by using a truncated version of the function $x\mapsto \frac{\partial }{\partial t}m(x,\theta ,t)$, and then, we insert such a robust estimator in the estimating equation corresponding to the minimum empirical divergence estimator. The truncated function is based on the multidimensional Huber function and contains a shift vector ${\tau }_{\theta }$ and a scale matrix $A_{\theta }$ to calibrate $t_{\theta }$ and, thus, $t_{\theta }$, which realizes the supremum in the duality formula, will also be the solution of a new equation based on the new truncated function.

For simplicity, for fixed $\theta \in \mathsf{\Theta }$, we also use the notation $m_{\theta }(x,t):=m(x,\theta ,t)$. With this notation, $t_{\theta }=t_{\theta }\left(P_0\right)$ defined in (16) is the unique solution of the equation:

$$\int \frac{\partial }{\partial t}{m}_{\theta }(x,t_{\theta }\left(P_0\right))d{P}_0\left(x\right)=0.$$

(26)

Consider the system

$$\int \frac{\partial }{\partial t}{m}_{\theta }(y,t)d{P}_0\left(y\right)=0$$

(27)

$$\int H_c(A[\frac{\partial }{\partial t}{m}_{\theta }(y,t)-\tau ])d{P}_0\left(y\right)=0$$

(28)

$$\int H_c(A[\frac{\partial }{\partial t}{m}_{\theta }(y,t)-\tau ])H_c{(A[\frac{\partial }{\partial t}{m}_{\theta }(y,t)-\tau ])}^{\top }d{P}_0\left(y\right)=I_{\ell +1}$$

(29)

where

$$H_c\left(y\right):=\left\{\begin{array}{c}y·min\left(1,\frac{c}{\parallel y\parallel }\right)\mathrm{if}y\ne 0\hfill \\ 0\mathrm{if}y=0\hfill \end{array}\right.$$

(30)

is the multidimensional Huber function, with $c>0$, $I_{\ell +1}$ the identity matrix, A is a $(l+1)\times (l+1)$ matrix, and $\tau \in {\mathbb{R}}^{l+1}$. For fixed $\theta$, this system admits a unique solution $(t,A,\tau )=(t_{\theta }\left(P_0\right),A_{\theta }\left(P_0\right),{\tau }_{\theta }\left(P_0\right))$ (according to [18], p. 17).

The multidimensional Huber function is useful to define robust estimators; it transforms each point outside a hypersphere of c radius to the nearest point of it and leaves those inside unchanged (see [26], p. 239, [27]). By applying the multidimensional Huber function to the function $y\mapsto \frac{\partial }{\partial t}{m}_{\theta }(y,t)$, together with considering the scale matrix $A_{\theta }$ and the shift vector ${\tau }_{\theta }$, a modification is produced there, where the norm exceeds the bound c, and in the meantime, the original $t_{\theta }$ remains the solution of the equation based on the new truncated function. For parametric models, the multidimensional Huber function was also used in other contexts, for example to define optimal B${}_s$-robust estimators or optimal B${}_i$-robust estimators (see [26], p. 244).

The above arguments can be used for each probability measure P from the moment condition model ${\mathcal{M}}^1$. This context allows defining the truncated version of the function $y\mapsto \frac{\partial }{\partial t}{m}_{\theta }(y,t)$, which we denote by ${\psi }_{\theta }(y,t)$, such that the original $t_{\theta }\left(P_0\right)$, the solution of Equation (26), is also the solution of the equation $\int {\psi }_{\theta }(y,t_{\theta }\left(P_0\right))d{P}_0\left(y\right)=0$.

For $\theta$ fixed and P a probability measure, the equation $\int \frac{\partial }{\partial t}{m}_{\theta }(y,t)dP\left(y\right)=0$ has a unique solution $t=t_{\theta }\left(P\right)\in {\mathsf{\Lambda }}_{\theta }\left(P\right)$ assuring the supremum in the dual form of the divergence $D_{\phi }({\mathcal{M}}_{\theta },P)$ (see [12]). For each t, we define the $A_{\theta }\left(t\right)$ and ${\tau }_{\theta }\left(t\right)$ solutions of the system:

$$\int H_c(A_{\theta }\left(t\right)[\frac{\partial }{\partial t}{m}_{\theta }(y,t)-{\tau }_{\theta }\left(t\right)])dP\left(y\right)=0$$

(31)

$$\int H_c(A_{\theta }\left(t\right)[\frac{\partial }{\partial t}{m}_{\theta }(y,t)-{\tau }_{\theta }\left(t\right)])H_c{(A_{\theta }\left(t\right)[\frac{\partial }{\partial t}{m}_{\theta }(y,t)-{\tau }_{\theta }\left(t\right)])}^{\top }dP\left(y\right)=I_{\ell +1}.$$

(32)

We define a new estimator ${\widehat{t}}_{\theta }^c$ of $t_{\theta }=t_{\theta }\left(P_0\right)$, as a Z-estimator corresponding to the $\psi$-function:

$${\psi }_{\theta }(x,t):=H_c(A_{\theta }\left(t\right)[\frac{\partial }{\partial t}{m}_{\theta }(x,t)-{\tau }_{\theta }\left(t\right)]);$$

(33)

more precisely, ${\widehat{t}}_{\theta }^c$ is defined by

$$\int {\psi }_{\theta }(y,{\widehat{t}}_{\theta }^c)d{P}_n\left(y\right)=0\mathrm{or}\sum _{i=1}^n{H}_c(A_{\theta }\left({\widehat{t}}_{\theta }^c\right)[\frac{\partial }{\partial t}{m}_{\theta }(X_i,{\widehat{t}}_{\theta }^c)-{\tau }_{\theta }\left({\widehat{t}}_{\theta }^c\right)])=0,$$

(34)

the theoretical counterpart of this estimating equation being

$$\int {\psi }_{\theta }(y,t_{\theta }\left(P_0\right))d{P}_0\left(y\right)=0.$$

(35)

For a given probability measure P, the statistical functional $t_{\theta }^c\left(P\right)$ associated with the estimator ${\widehat{t}}_{\theta }^c$, whenever it exists, is defined by

$$\int {\psi }_{\theta }(y,t_{\theta }^c\left(P\right))dP\left(y\right)=\int H_c(A_{\theta }\left(t_{\theta }^c\left(P\right)\right)[\frac{\partial }{\partial t}{m}_{\theta }(y,t_{\theta }^c\left(P\right))-{\tau }_{\theta }\left(t_{\theta }^c\left(P\right)\right)])dP\left(y\right)=0.$$

(36)

Note that

$$t_{\theta }^c\left(P_0\right)=t_{\theta }\left(P_0\right),$$

(37)

by construction.

Remark 1.

We notice a similarity between the Z-estimator defined in (34) and the classical optimal Bs-robust estimator for parametric models from [26]. In the case of the parametric models, the M-estimator corresponding to the ψ-function (33), but defined for the classical score function $\frac{\partial }{\partial t}(ln{f}_t\left(x\right))=\frac{\frac{\partial }{\partial t}{f}_t\left(x\right)}{f_t\left(x\right)}$ instead of the function $\frac{\partial }{\partial t}{m}_{\theta }(x,t)$ (inclusively in the system (31) and (32) defining $A_{\theta }\left(t\right)$ and ${\tau }_{\theta }\left(t\right)$), is the classical optimal Bs-robust estimator ($f_t\left(x\right)$ denotes the density corresponding to a parametric model indexed by the parameter t). The classical optimal Bs-robust estimator for parametric models has the optimal property that minimizes a measure of the asymptotic mean-squared error, among all the Fisher-consistent estimators with a self-standardized sensitivity smaller than the positive constant c.

In the following, for a given divergence, using the estimators ${\widehat{t}}_{\theta }^c$ for $t_{\theta }\left(P_0\right)$, we constructed new estimators of the parameter ${\theta }_0$ of the model. In Section 3.3, we prove that all the estimators ${\widehat{t}}_{\theta }^c$ are robust, and this property will be transferred to the new estimators that we define for the parameter ${\theta }_0$.

To define new estimators for ${\theta }_0$, we used the dual representation (15) of the divergence $D_{\phi }({\mathcal{M}}_{\theta },P_0)$. Since

$$\begin{array}{ccc}\hfill {\theta }_0=arg\underset{\theta \in \mathsf{\Theta }}{inf}{D}_{\phi }({\mathcal{M}}_{\theta },P_0)& =& arg\underset{\theta \in \mathsf{\Theta }}{inf}\underset{t\in {\mathsf{\Lambda }}_{\theta }}{sup}\int m_{\theta }(y,t)d{P}_0\left(y\right)\hfill \end{array}$$

(38)

$$\begin{array}{ccc}& =& arg\underset{\theta \in \mathsf{\Theta }}{inf}\int m_{\theta }(y,t_{\theta }\left(P_0\right))d{P}_0\left(y\right),\hfill \end{array}$$

(39)

$\theta ={\theta }_0$ is the solution of the equation:

$$\int \frac{\partial }{\partial \theta }\left[m(y,\theta ,t(\theta ,P_0))\right]d{P}_0\left(y\right)=0,$$

(40)

where we used the notation $t(\theta ,P):=t_{\theta }\left(P\right)$. Equation (40) may be written as

$$\int \frac{\partial }{\partial \theta }m(y,{\theta }_0,t({\theta }_0,P_0))d{P}_0\left(y\right)+\frac{\partial }{\partial \theta }t{({\theta }_0,P_0)}^{\top }\int \frac{\partial }{\partial t}m(y,{\theta }_0,t({\theta }_0,P_0))d{P}_0\left(y\right)=0.$$

(41)

On the basis of the definition of $t_{\theta }\left(P_0\right)=t(\theta ,P_0)$, for $\theta ={\theta }_0$, we have

$$\int \frac{\partial }{\partial t}m(y,{\theta }_0,t({\theta }_0,P_0))d{P}_0\left(y\right)=0;$$

(42)

therefore, we deduce that $\theta ={\theta }_0$ is the solution of equation:

$$\int \frac{\partial }{\partial \theta }m(y,\theta ,t(\theta ,P_0))d{P}_0\left(y\right)=0.$$

(43)

Using (37), namely $t^c(\theta ,P_0)=t(\theta ,P_0)$, we obtain that $\theta ={\theta }_0$ is in fact the solution of equation:

$$\int \frac{\partial }{\partial \theta }m(y,\theta ,t^c(\theta ,P_0))d{P}_0\left(y\right)=0.$$

(44)

Then, we define a new estimator ${\widehat{\theta }}_{\phi }^c$ of ${\theta }_0$, as a plug-in estimator solution of the equation:

$$\int \frac{\partial }{\partial \theta }m(y,{\widehat{\theta }}_{\phi }^c,t^c({\widehat{\theta }}_{\phi }^c,P_n))d{P}_n\left(y\right)=0.$$

(45)

For a probability measure P, the statistical functional $T^c$ corresponding to the estimator ${\widehat{\theta }}_{\phi }^c$, whenever it exists, is defined by

$$\int \frac{\partial }{\partial \theta }m(y,T^c\left(P\right),t^c(T^c\left(P\right),P))dP\left(y\right)=0.$$

(46)

The functional $T^c$ is Fisher-consistent, because

$$T^c\left(P_0\right)={\theta }_0.$$

(47)

This equality is obtained by using (46) for $P=P_0$, the fact that $t^c(T^c\left(P_0\right),P_0)=t(T^c\left(P_0\right),P_0)$, and the definition of $t_{\theta }\left(P_0\right)=t(\theta ,P_0)$ for $\theta =T^c\left(P_0\right)$, all these leading to

$$\int \frac{\partial }{\partial \theta }m(y,T^c\left(P_0\right),t(T^c\left(P_0\right),P_0))d{P}_0\left(y\right)+\frac{\partial }{\partial \theta }t{(T^c\left(P_0\right),P_0)}^{\top }\int \frac{\partial }{\partial t}m(y,T^c\left(P_0\right),t(T^c\left(P_0\right),P_0))d{P}_0\left(y\right)=0.$$

(48)

Since ${\theta }_0$ is the unique solution of Equation (41) and, according to (48), $T^c\left(P_0\right)$ would be another solution to the same equation, we deduce (47).

From (34) and (45), we have

$$\begin{array}{c}\int {\psi }_{{\widehat{\theta }}_{\phi }^c}(y,t^c({\widehat{\theta }}_{\phi }^c,P_n))d{P}_n\left(y\right)=0,\hfill \\ \int \frac{\partial }{\partial \theta }m(y,{\widehat{\theta }}_{\phi }^c,t^c({\widehat{\theta }}_{\phi }^c,P_n))d{P}_n\left(y\right)=0,\hfill \end{array}$$

and then,

$$\begin{array}{c}\int \psi (y,{\widehat{\theta }}_{\phi }^c,{\widehat{t}}_{{\widehat{\theta }}_{\phi }^c}^c)d{P}_n\left(y\right)=0,\hfill \\ \int \frac{\partial }{\partial \theta }m(y,{\widehat{\theta }}_{\phi }^c,{\widehat{t}}_{{\widehat{\theta }}_{\phi }^c}^c)d{P}_n\left(y\right)=0,\hfill \end{array}$$

with $\psi (y,\theta ,t):={\psi }_{\theta }(y,t)$. The couple of estimators ${\widehat{\theta }}_{\phi }^c,{\widehat{t}}_{{\widehat{\theta }}_{\phi }^c}^c$ can be viewed as a Z-estimator solution of the above system. Denoting

$$\mathsf{\Psi }(y,\theta ,t):={(\psi {(y,\theta ,t)}^{\top },{(\frac{\partial }{\partial \theta }m(y,\theta ,t))}^{\top })}^{\top },$$

(49)

the Z-estimators ${\widehat{\theta }}_{\phi }^c,{\widehat{t}}_{{\widehat{\theta }}_{\phi }^c}^c$ are the solutions of the system:

$$\int \mathsf{\Psi }(y,{\widehat{\theta }}_{\phi }^c,{\widehat{t}}_{{\widehat{\theta }}_{\phi }^c}^c)d{P}_n\left(y\right)=0,$$

(50)

and the theoretical counterpart is given by

$$\int \mathsf{\Psi }(y,{\theta }_0,t_{{\theta }_0})d{P}_0\left(y\right)=0.$$

(51)

3.2. Asymptotic Properties

In this section, we establish the consistency and the asymptotic distributions for the estimators ${\widehat{\theta }}_{\phi }^c$ and ${\widehat{t}}_{{\widehat{\theta }}_{\phi }^c}^c$. In order to prove the consistency of the estimators, we adopted the results from the general theory of Z-estimators as presented for example in [28]. Then, using the consistency of the estimators, as well as supplementary conditions, we proved that the asymptotic distributions of the estimators are multivariate normal:

Assumption A1.

(a) 

There exist compact neighbourhoods $V_{{\theta }_0}$ of ${\theta }_0$ and $V_{t_{{\theta }_0}}$ of $t_{{\theta }_0}$ such that

$$\int \underset{\theta \in V_{{\theta }_0},t\in V_{t_{{\theta }_0}}}{sup}\parallel \mathsf{\Psi }(y,\theta ,t)\parallel d{P}_0\left(y\right)<\infty .$$

(b) 

For any positive ε, the following condition holds

$$\underset{(\theta ,t)\in M}{inf}\parallel \int \mathsf{\Psi }(y,\theta ,t)\parallel d{P}_0\left(y\right)>0=\parallel \int \mathsf{\Psi }(y,{\theta }_0,t_{{\theta }_0})d{P}_0\left(y\right)\parallel ,$$

where $M:=\{(\theta ,t)\mathrm{s}.\mathrm{t}.\parallel (\theta ,t)-({\theta }_0,t_{{\theta }_0})\parallel >\epsilon \}$.

Proposition 1.

Under Assumption 1, ${\widehat{\theta }}_{\phi }^c$ converges in probability to ${\theta }_0$ and ${\widehat{t}}_{{\widehat{\theta }}_{\phi }^c}^c$ converges in probability to $t_{{\theta }_0}$:

Assumption 2.

(a) 

Both estimators ${\widehat{\theta }}_{\phi }^c$ and ${\widehat{t}}_{{\widehat{\theta }}_{\phi }^c}^c$ converge in probability to ${\theta }_0$ and $t_{{\theta }_0}$, respectively.

(b) 

The function $(\theta ,t)\mapsto \psi (x,\theta ,t)$ is ${\mathcal{C}}^2$ on some neighbourhood $V({\theta }_0,t_{{\theta }_0})$ for all x ($P_0$ a.s.), and the partial derivatives of order two of the functions $\{(\theta ,t)\mapsto \psi (x,\theta ,t);(\theta ,t)\in V({\theta }_0,t_{{\theta }_0})\}$ are dominated by some $P_0$-integrable function $H_1\left(x\right)$.

(c) 

The function $(\theta ,t)\mapsto m(x,\theta ,t)$ is ${\mathcal{C}}^3$ on some neighbourhood $U({\theta }_0,t_{{\theta }_0})$ for all x ($P_0$ a.s.), and the partial derivatives of order three of the functions $\{(\theta ,t)\mapsto m(x,\theta ,t);(\theta ,t)\in U({\theta }_0,t_{{\theta }_0})\}$ are dominated by some $P_0$-integrable function $H_2\left(x\right)$.

(d) 

$\int \parallel \frac{\partial }{\partial \theta }m(y,{\theta }_0,t_{{\theta }_0}){\parallel }^{2}d{P}_0\left(y\right)$ is finite, and the matrix:

$$S:=\left(\begin{array}{cc}{S}_{11}& S_{12}\\ S_{21}& S_{22}\end{array}\right),$$

(52)

with $S_{11}:={(\int \frac{\partial }{\partial t}\psi (y,{\theta }_0,t_{{\theta }_0})d{P}_0\left(y\right))}^{\top }$, $S_{12}:={(\int \frac{\partial }{\partial \theta }\psi (y,{\theta }_0,t_{{\theta }_0})d{P}_0\left(y\right))}^{\top }$, $S_{21}:={(\int \frac{{\partial }^2}{\partial \theta \partial t}m(y,{\theta }_0,t_{{\theta }_0})d{P}_0\left(y\right))}^{\top }$ and $S_{22}:=\int \frac{{\partial }^2}{{\partial }^2\theta }m(y,{\theta }_0,t_{{\theta }_0})d{P}_0\left(y\right)$, exists and is invertible.

Proposition 2.

Let $P_0$ belong to the model ${\mathcal{M}}^1$, and suppose that Assumption 2 holds. Then, both $\sqrt{n}({\widehat{\theta }}_{\phi }^c-{\theta }_0)$ and $\sqrt{n}({\widehat{t}}_{{\widehat{\theta }}_{\phi }^c}^c-t_{{\theta }_0})$ converge in distribution to a centred multivariate normal variable with covariance matrices given by

$$\left[{\left[S_{21}{S}_{11}^{-1}{S}_{12}\right]}^{-1}{S}_{21}{S}_{11}^{-1}\right]\times {\left[{\left[S_{21}{S}_{11}^{-1}{S}_{12}\right]}^{-1}{S}_{21}{S}_{11}^{-1}\right]}^{\top },$$

(53)

and

$$[S_{11}^{-1}-S_{11}^{-1}{S}_{12}{\left[S_{21}{S}_{11}^{-1}{S}_{12}\right]}^{-1}{S}_{21}{S}_{11}^{-1}]\times {[S_{11}^{-1}-S_{11}^{-1}{S}_{12}{\left[S_{21}{S}_{11}^{-1}{S}_{12}\right]}^{-1}{S}_{21}{S}_{11}^{-1}]}^{\top }.$$

(54)

The condition of Type (a) from Assumption 1 is usually considered to apply the uniform law of large numbers. For many choices of divergence (for example, those from the Cressie–Read family), the function $\mathsf{\Psi }$ is continuous in $(\theta ,t)$, and consequently, this condition is verified. The second condition from Assumption 1 is imposed for the uniqueness of $({\theta }_0,t_{{\theta }_0})$ as a solution of the equation and is verified, for example, whenever $\mathsf{\Psi }$ is continuous and the parameter space is compact ([28], p. 46). Furthermore, the conditions of Type (b)–(d), included in Assumption 2, are often imposed in order to apply the law of large numbers or the central limit theorem and can be verified for the functions appearing in the definitions of estimators proposed in the present paper.

3.3. Influence Functions and Robustness

In this section, we derive the influence functions of the estimators ${\widehat{t}}_{\theta }^c$ and ${\widehat{\theta }}_{\phi }^c$ and prove their B-robustness. The corresponding statistical functionals are defined by (36) and (46), respectively.

Recall that, a map T, defined on a set of probability measures and parameter-space-valued, is a statistical functional corresponding to an estimator $\widehat{\theta }$ of the parameter ${\theta }_0$ from the model $P_0$, if $\widehat{\theta }=T\left(P_n\right)$, $P_n$ being the empirical measure corresponding to the sample. The influence function of T at $P_0$ is defined by

$$\mathrm{IF}(x;T,P_0):={\left.\frac{\partial T\left({\tilde{P}}_{\epsilon x}\right)}{\partial \epsilon }\right|}_{\epsilon =0},$$

where ${\tilde{P}}_{\epsilon x}:=(1-\epsilon )P_0+\epsilon {\delta }_x$, ${\delta }_x$ being the Dirac measure. An unbounded influence function implies an unbounded asymptotic bias of a statistic under single-point contamination of the model. Therefore, a natural robustness requirement on a statistical functional is the boundedness of its influence function. Whenever the influence function is bounded with respect to x, the corresponding estimator is called B-robust [26].

Proposition 3.

For fixed θ, the influence function of the functional $t_{\theta }^c$ is given by

$$\mathrm{IF}(x;t_{\theta }^c,P_0)=-{\left\{\int \frac{\partial }{\partial t}{\psi }_{\theta }(y,t_{\theta }\left(P_0\right))d{P}_0\left(y\right)\right\}}^{-1}·{\psi }_{\theta }(x,t_{\theta }\left(P_0\right)).$$

(55)

Proposition 4.

The influence function of the functional $T^c$ is given by

$$\begin{array}{c}\mathrm{IF}(x;T^c,P_0)={\left\{{\left[\frac{\partial }{\partial \theta }\overline{g}(y,{\theta }_0)d{P}_0\left(y\right)\right]}^{\top }{\left[\int \overline{g}(y,{\theta }_0)\overline{g}{(y,{\theta }_0)}^{\top }d{P}_0\left(y\right)\right]}^{-1}\left[\frac{\partial }{\partial \theta }\overline{g}(y,{\theta }_0)d{P}_0\left(y\right)\right]\right\}}^{-1}\\ ·{\left[\frac{\partial }{\partial \theta }\overline{g}(y,{\theta }_0)d{P}_0\left(y\right)\right]}^{\top }\frac{1}{{\phi }^{\prime \prime }\left(1\right)}\mathrm{IF}(x;t_{{\theta }_0}^c,P_0).\end{array}$$

(56)

On the basis of Propositions 3 and 4, since $x\mapsto {\psi }_{\theta }(x,t_{\theta }\left(P_0\right))$ is bounded, all the estimators ${\widehat{\theta }}_{\phi }^c$ are B-robust.

4. Conclusions

We introduced a class of robust Z-estimators for moment condition models. These new estimators can be seen as robust alternatives for the minimum empirical divergence estimators. By using truncated functions based on the multidimensional Huber function, we defined robust estimators of the element that realizes the supremum in the dual form of the divergence, as well as new robust estimators for the parameter of the model. The asymptotic properties were proven, including the consistency and the limit laws. The influence functions for all the proposed estimators are bounded; therefore, these estimators are B-robust. The truncated function that we used to define the new robust Z-estimators contains functions implicitly defined, for which analytic forms are not available. The implementation of the estimation method will be addressed in a future research study. The idea of using the multidimensional Huber function, together with a scale matrix and a shift vector, to create a bounded version of the function corresponding to the estimating equation for the parameter of interest, could be considered in other contexts as well and would lead to new robust Z-estimators. As one of the Referees suggested, some other bounded functions could be used to define new robust Z-estimators for moment condition models. For example, the Tukey biweight function used together with a norm inside, in order to be appropriate to be applied to functions with vector values, could also be considered. Again, the original parameter of interest should remain the solution of the estimating equation based on the new bounded function. Such an idea is interesting to be analysed in future studies, in order to provide new robust versions of minimum empirical divergence estimators or robust Z-estimators in other contexts.