Robustness Property of Robust-BD Wald-Type Test for Varying-Dimensional General Linear Models

Abstract

An important issue for robust inference is to examine the stability of the asymptotic level and power of the test statistic in the presence of contaminated data. Most existing results are derived in finite-dimensional settings with some particular choices of loss functions. This paper re-examines this issue by allowing for a diverging number of parameters combined with a broader array of robust error measures, called “robust-$\mathrm{BD}$”, for the class of “general linear models”. Under regularity conditions, we derive the influence function of the robust-$\mathrm{BD}$ parameter estimator and demonstrate that the robust-$\mathrm{BD}$ Wald-type test enjoys the robustness of validity and efficiency asymptotically. Specifically, the asymptotic level of the test is stable under a small amount of contamination of the null hypothesis, whereas the asymptotic power is large enough under a contaminated distribution in a neighborhood of the contiguous alternatives, thus lending supports to the utility of the proposed robust-$\mathrm{BD}$ Wald-type test.

1. Introduction

The class of varying-dimensional “general linear models” [1], including the conventional generalized linear model ($\mathrm{GLM}$ in [2]), is flexible and powerful for modeling a large variety of data and plays an important role in many statistical applications. In the literature, it has been extensively studied that the conventional maximum likelihood estimator for the $\mathrm{GLM}$ is nonrobust; for example, see [3,4]. To enhance the resistance to outliers in applications, many efforts have been made to obtain robust estimators. For example, Noh et al. [5] and Künsch et al. [6] developed robust estimator for the $\mathrm{GLM}$, and Stefanski et al. [7], Bianco et al. [8] and Croux et al. [9] studied robust estimation for the logistic regression model with the deviance loss as the error measure.

Besides robust estimation for the $\mathrm{GLM}$, robust inference is another important issue, which, however, receives relatively less attention. Basically, the study of robust testing includes two aspects: (i) establishing the stability of the asymptotic level under small departures from the null hypothesis (i.e., robustness of “validity”); and (ii) demonstrating that the asymptotic power is sufficiently large under small departures from specified alternatives (i.e., robustness of “efficiency”). In the literature, robust inference has been conducted for different models. For example, Heritier et al. [10] studied the robustness properties of the Wald, score and likelihood ratio tests based on M estimators for general parametric models. Cantoni et al. [11] developed a test statistic based on the robust deviance, and conducted robust inference for the $\mathrm{GLM}$ using quasi-likelihood as the loss function. A robust Wald-type test for the logistic regression model is studied in [12]. Ronchetti et al. [13] concerned the robustness property for the generalized method of moments estimators. Basu et al. [14] proposed robust tests based on the density power divergence (DPD) measure for the equality of two normal means. Robust tests for parameter change have been studied using the density-based divergence method in [15,16]. However, the aforementioned methods based on the $\mathrm{GLM}$ mostly focus on situations where the number of parameters is fixed and the loss function is specific.

Zhang et al. [1] developed robust estimation and testing for the “general linear model” based on a broader array of error measures, namely Bregman divergence, allowing for a diverging number of parameters. The Bregman divergence includes a wide class of error measures as special cases, e.g., the (negative) quasi-likelihood in regression, the deviance loss and exponential loss in machine learning practice, among many other commonly used loss functions. Zhang et al. [1] studied the consistency and asymptotic normality of their proposed robust-$\mathrm{BD}$ parameter estimator and demonstrated the asymptotic distribution of the Wald-type test constructed from robust-$\mathrm{BD}$ estimators. Naturally, it remains an important issue to examine the robustness property of the robust-$\mathrm{BD}$ Wald-type test [1] in the varying-dimensional case, i.e., whether the test still has stable asymptotic level and power, in the presence of contaminated data.

This paper aims to demonstrate the robustness property of the robust-$\mathrm{BD}$ Wald-type test in [1]. Nevertheless, it is a nontrivial task to address this issue. Although the local stability for the Wald-type tests have been established for the M estimators [10], generalized method of moment estimators [13], minimum density power divergence estimator [17] and general M estimators under random censoring [18], their results for finite-dimensional settings are not directly applicable to our situations with a diverging number of parameters. Under certain regularity conditions, we provide rigorous theoretical derivation for robust testing based on the Wald-type test statistic. The essential results are approximations of the asymptotic level and power under contaminated distributions of the data in a small neighborhood of the null and alternative hypotheses, respectively.

• Specifically, we show in Theorem 1 that, if the influence function of the estimator is bounded, then the asymptotic level of the test is also bounded under a small amount of contamination.
• We also demonstrate in Theorem 2 that, if the contamination belongs to a neighborhood of the contiguous alternatives, then the asymptotic power is also stable.

Hence, we contribute to establish the robustness of validity and efficiency for the robust-$\mathrm{BD}$ Wald-type test for the “general linear model” with a diverging number of parameters.

The rest of the paper is organized as follows. Section 2 reviews the Bregman divergence ($\mathrm{BD}$), robust-$\mathrm{BD}$ estimation and the Wald-type test statistic proposed in [1]. Section 3 derives the influence function of the robust-$\mathrm{BD}$ estimator and studies the robustness properties of the asymptotic level and power of the Wald-type test under a small amount of contamination. Section 4 conducts the simulation studies. The technical conditions and proofs are given in Appendix A. A list of notations and symbols is provided in Appendix B.

We will introduce some necessary notations. In the following, C and c are generic finite constants which may vary from place to place, but do not depend on the sample size n. Denote by ${\mathrm{E}}_K(·)$ the expectation with respect to the underlying distribution K. For a positive integer q, let ${\mathbf{0}}_q={(0,\dots ,0)}^T\in {\mathbb{R}}^q$ be a $q\times 1$ zero vector and ${\mathbf{I}}_q$ be the $q\times q$ identity matrix. For a vector $\mathit{v}={(v_1,\dots ,v_q)}^T\in {\mathbb{R}}^q$, the $L_1$ norm is ${\parallel \mathit{v}\parallel }_1={\sum }_{i=1}^q\left|v_i\right|$, $L_2$ norm is ${\parallel \mathit{v}\parallel }_2={({\sum }_{i=1}^q{v}_i^2)}^{1/2}$ and the $L_{\infty }$ norm is ${\parallel \mathit{v}\parallel }_{\infty }={max}_{i=1,\dots ,q}\left|v_i\right|$. For a $q\times q$ matrix A, the $L_2$ and Frobenius norms of A are ${\parallel A\parallel }_2={\{{\lambda }_{max}(A^{T}A)\}}^{1/2}$ and ${\parallel A\parallel }_F=\sqrt{\mathrm{tr}(A{A}^T)}$, respectively, where ${\lambda }_{max}(·)$ denotes the largest eigenvalue of a matrix and $\mathrm{tr}(·)$ denotes the trace of a matrix.

2. Review of Robust-$\mathrm{BD}$ Estimation and Inference for “General Linear Models”

This section briefly reviews the robust-$\mathrm{BD}$ estimation and inference methods for the “general linear model” developed in [1]. Let $\{({\mathit{X}}_{n1},Y_1),\dots ,({\mathit{X}}_{nn},Y_n)\}$ be $\mathrm{i}.\mathrm{i}.\mathrm{d}.$ observations from some underlying distribution $({\mathit{X}}_n,Y)$ with ${\mathit{X}}_n={(X_1,\dots ,X_{p_n})}^T\in {\mathbb{R}}^{p_n}$ the explanatory variables and Y the response variable. The dimension $p_n$ is allowed to diverge with the sample size n. The “general linear model” is given by

$$\begin{array}{c}\hfill m({\mathit{x}}_n)\equiv \mathrm{E}(Y\mid {\mathit{X}}_n={\mathit{x}}_n)=F^{-1}({\tilde{\mathit{x}}}_n^T{\tilde{\mathit{\beta }}}_{n,0}),\end{array}$$

(1)

and

$$\begin{array}{c}\hfill \mathrm{var}(Y\mid {\mathit{X}}_n={\mathit{x}}_n)=V(m({\mathit{x}}_n)),\end{array}$$

(2)

where F is a known link function, ${\tilde{\mathit{\beta }}}_{n,0}\in {\mathbb{R}}^{p_n+1}$ is the vector of unknown true regression parameters, ${\tilde{\mathit{x}}}_n={(1,{\mathit{x}}_n^T)}^T$ and $V(·)$ is a known function. Note that the conventional generalized linear model ($\mathrm{GLM}$) satisfying Equations (1) and (2) assumes that $Y\mid {\mathit{X}}_n={\mathit{x}}_n$ follows a particular distribution in the exponential family. However, our “general linear model” does not require explicit form of distributions of the response. Hence, the “general linear model” includes the $\mathrm{GLM}$ as a special case. For notational simplicity, denote ${\mathit{Z}}_n={({\mathit{X}}_n^T,Y)}^T$ and ${\tilde{\mathit{Z}}}_n={({\tilde{\mathit{X}}}_n^T,Y)}^T$.

Bregman divergence ($\mathrm{BD}$) is a class of error measures, which is introduced in [19] and covers a wide range of loss functions. Specifically, Bregman divergence is defined as a bivariate function,

$$\begin{array}{c}\hfill Q_q(\nu ,\mu )=-q(\nu )+q(\mu )+(\nu -\mu )q^{\prime }(\mu ),\end{array}$$

where $q(·)$ is the concave generating q-function. For example, $q(\mu )=a\mu -{\mu }^2$ for a constant a corresponds to the quadratic loss $Q_a(Y,\mu )={(Y-\mu )}^2$. For a binary response variable Y, $q(\mu )=min\{\mu ,1-\mu \}$ gives the misclassification loss $Q_q(Y,\mu )=\mathrm{I}\{Y\ne \mathrm{I}(\mu >0.5)\}$; $q(\mu )=-2\{\mu log(\mu )+(1-\mu )log(1-\mu )\}$ gives Bernoulli deviance loss $Q_q(Y,\mu )=-2\{Ylog(\mu )+(1-Y)log(1-\mu )\}$; $q(\mu )=2min\{\mu ,1-\mu \}$ gives the hinge loss $Q_q(Y,\mu )=max\{1-(2Y-1)\mathrm{sign}(\mu -0.5),0\}$ for the support vector machine; $q(\mu )=2{\{\mu (1-\mu )\}}^{1/2}$ yields the exponential loss $Q_q(Y,\mu )=exp[-(Y-0.5)log\{\mu /(1-\mu )\}]$ used in AdaBoost [20]. Furthermore, Zhang et al. [21] showed that if

$$\begin{array}{c}\hfill q(\mu )={\int }_a^{\mu }\frac{s-\mu }{V(s)}ds,\end{array}$$

(3)

where a is a finite constant such that the integral is well-defined, then $Q_q(y,\mu )$ is the “classical (negative) quasi-likelihood” function $-{\mathtt{Q}}_{\mathrm{QL}}(y,\mu )$ with $\partial {\mathtt{Q}}_{\mathrm{QL}}(y,\mu )/\partial \mu =(y-\mu )/V(\mu )$.

To obtain a robust estimator based on $\mathrm{BD}$, Zhang et al. [1] developed the robust-$\mathrm{BD}$ loss function

$$\begin{array}{c}\hfill {\rho }_q(y,\mu )={\int }_y^{\mu }\psi (r(y,s))\{q^{″}(s)\sqrt{V(s)}\}ds-G(\mu ),\end{array}$$

(4)

where $\psi (·)$ is a bounded odd function, such as the Huber $\psi$-function [22], $r(y,s)=(y-s)/\sqrt{V(s)}$ denotes the Pearson residual and $G(\mu )$ is the bias-correction term satisfying

$$\begin{array}{c}\hfill G^{\prime }(\mu )=G_1^{\prime }(\mu )\{q^{″}(\mu )\sqrt{V(\mu )}\},\end{array}$$

with

$$\begin{array}{c}\hfill G_1^{\prime }(m({\mathit{x}}_n))=\mathrm{E}\{\psi (r(Y,m({\mathit{x}}_n)))\mid {\mathit{X}}_n={\mathit{x}}_n\}.\end{array}$$

Based on robust-$\mathrm{BD}$, the estimator of ${\tilde{\mathit{\beta }}}_{n,0}$ proposed in [1] is defined as

$$\begin{array}{c}\hfill \widehat{\tilde{\mathit{\beta }}}=arg\underset{\tilde{\mathit{\beta }}}{min}\left\{\frac{1}{n}\sum _{i=1}^n{\rho }_q(Y_i,F^{-1}({\tilde{\mathit{X}}}_{ni}^T\tilde{\mathit{\beta }}))w({\mathit{X}}_{ni})\right\},\end{array}$$

(5)

where $w(·)\ge 0$ is a known bounded weight function which downweights the high leverage points.

In [11], the “robust quasi-likelihood estimator” of ${\tilde{\mathit{\beta }}}_{n,0}$ is formulated according to the “robust quasi-likelihood function” defined as

$$\begin{array}{cc}& {\mathtt{Q}}_{\mathrm{RQL}}({\mathit{x}}_n,y,\mu )\hfill \\ =& \left\{{\int }_{{\mu }_0}^{\mu }\psi (r(y,s))/\sqrt{V(s)}ds\right\}w({\mathit{x}}_n)-\frac{1}{n}\sum _{j=1}^n{\int }_{{\mu }_0}^{{\mu }_j}\left[\mathrm{E}\{\psi (r(Y_j,s))|{\mathit{X}}_{nj}\}/\sqrt{V(s)}ds\right]w({\mathit{X}}_{nj}),\hfill \end{array}$$

where $\mu =F^{-1}({\tilde{\mathit{x}}}_n^T\tilde{\mathit{\beta }})$ and ${\mu }_j={\mu }_j(\tilde{\mathit{\beta }})=F^{-1}({\tilde{\mathit{X}}}_{nj}^T\tilde{\mathit{\beta }})$, $j=1,\dots ,n$. To describe the intuition of the “robust-$\mathrm{BD}$”, we use the following diagram from [1], which illustrates the relation among the “robust-$\mathrm{BD}$”, “classical-$\mathrm{BD}$”, “robust quasi-likelihood” and “classical (negative) quasi-likelihood”.

For the robust-$\mathrm{BD}$, assume that

$$\begin{array}{c}\hfill {\mathrm{p}}_j(y;\theta )=\frac{{\partial }^j}{\partial {\theta }^j}{\rho }_q(y,F^{-1}(\theta )),j=0,1,\dots ,\end{array}$$

exist finitely up to any order required. For example, for $j=1$,

$$\begin{array}{c}\hfill {\mathrm{p}}_1(y;\theta )=\{\psi (r(y,\mu ))-G_1^{\prime }(\mu )\}\{q^{″}(\mu )\sqrt{V(\mu )}\}/F^{\prime }(\mu ),\end{array}$$

(6)

where $\mu =F^{-1}(\theta )$. Explicit expressions for ${\mathrm{p}}_j(y;\theta )$ ($j=2,3$) can be found in Equation (3.7) of [1]. Then, the estimation equation for $\widehat{\tilde{\mathit{\beta }}}$ is

$$\begin{array}{c}\hfill \frac{1}{n}\sum _{i=1}^n{\mathit{\psi }}_{\mathrm{RBD}}({\mathit{Z}}_{ni};\tilde{\mathit{\beta }})=\mathbf{0},\end{array}$$

where the score vector is

$$\begin{array}{c}\hfill {\mathit{\psi }}_{\mathrm{RBD}}({\mathit{z}}_n;\tilde{\mathit{\beta }})={\mathrm{p}}_1(y;\theta )w({\mathit{x}}_n){\tilde{\mathit{x}}}_n,\end{array}$$

(7)

with $\theta ={\tilde{\mathit{x}}}_n^T\tilde{\mathit{\beta }}$. The consistency and asymptotic normality of $\widehat{\tilde{\mathit{\beta }}}$ have been studied in [1]; see Theorems 1 and 2 therein.

Furthermore, to conduct statistical inference for the “general linear model”, the following hypotheses are considered,

$$\begin{array}{c}\hfill H_0:A_n{\tilde{\mathit{\beta }}}_{n,0}={\mathit{g}}_0\mathrm{versus}{H}_1:A_n{\tilde{\mathit{\beta }}}_{n,0}\ne {\mathit{g}}_0,\end{array}$$

(8)

where $A_n$ is a given $\mathrm{k}\times (p_n+1)$ matrix such that $A_n{A}_n^T\to \mathbb{G}$ with $\mathbb{G}$ being a $\mathrm{k}\times \mathrm{k}$ positive-definite matrix, and ${\mathit{g}}_0$ is a known $\mathrm{k}\times 1$ vector.

To perform the test of Equation (8), Zhang et al. [1] proposed the Wald-type test statistic,

$$\begin{array}{c}\hfill W_n=n{(A_n\widehat{\tilde{\mathit{\beta }}}-{\mathit{g}}_0)}^T{(A_n{\widehat{\mathbf{H}}}_n^{-1}{\widehat{\mathsf{\Omega }}}_n{\widehat{\mathbf{H}}}_n^{-1}{A}_n^T)}^{-1}(A_n\widehat{\tilde{\mathit{\beta }}}-{\mathit{g}}_0),\end{array}$$

(9)

constructed from the robust-$\mathrm{BD}$ estimator $\widehat{\tilde{\mathit{\beta }}}$ in Equation (5), where

$$\begin{array}{ccc}\hfill {\widehat{\mathsf{\Omega }}}_n& =& \frac{1}{n}\sum _{i=1}^n{\mathrm{p}}_1^2(Y_i;{\tilde{\mathit{X}}}_{ni}^T\widehat{\tilde{\mathit{\beta }}})w^2({\mathit{X}}_{ni}){\tilde{\mathit{X}}}_{ni}{\tilde{\mathit{X}}}_{ni}^T,\hfill \\ \hfill {\widehat{\mathbf{H}}}_n& =& \frac{1}{n}\sum _{i=1}^n{\mathrm{p}}_2(Y_i;{\tilde{\mathit{X}}}_{ni}^T\widehat{\tilde{\mathit{\beta }}})w({\mathit{X}}_{ni}){\tilde{\mathit{X}}}_{ni}{\tilde{\mathit{X}}}_{ni}^T.\hfill \end{array}$$

The asymptotic distributions of $W_n$ under the null and alternative hypotheses have been developed in [1]; see Theorems 4–6 therein.

On the other hand, the issue on the robustness of $W_n$, used for possibly contaminated data, remains unknown. Section 3 of this paper will address this issue with detailed derivations.

3. Robustness Properties of $W_n$ in Equation (9)

This section derives the influence function of the robust-$\mathrm{BD}$ Wald-type test and studies the influence of a small amount of contamination on the asymptotic level and power of the test. The proofs of the theoretical results are given in Appendix A.

Denote by $K_{n,0}$ the true distribution of ${\mathit{Z}}_n$ following the “general linear model” characterized by Equations (1) and (2). To facilitate the discussion of robustness properties, we consider the $ϵ$-contamination,

$$\begin{array}{c}\hfill K_{n,ϵ}=\left(1-\frac{ϵ}{\sqrt{n}}\right)K_{n,0}+\frac{ϵ}{\sqrt{n}}J,\end{array}$$

(10)

where J is an arbitrary distribution and $ϵ>0$ is a constant. Then, $K_{n,ϵ}$ is a contaminated distribution of ${\mathit{Z}}_n$ with the amount of contamination converging to 0 at rate $1/\sqrt{n}$. Denote by ${\mathbb{K}}_n$ the empirical distribution of ${\{{{\mathit{Z}}_n}_i\}}_{i=1}^n$.

For a generic distribution K of ${\mathit{Z}}_n$, define

$$\begin{array}{ccc}\hfill {\ell }_K(\tilde{\mathit{\beta }})& =& {\mathrm{E}}_K\{{\rho }_q(Y,F^{-1}({\tilde{\mathit{X}}}_n^T\tilde{\mathit{\beta }}))w({\mathit{X}}_n)\},\hfill \\ \hfill {\mathcal{S}}_K& =& \{\tilde{\mathit{\beta }}:{\mathrm{E}}_K\{{\mathit{\psi }}_{\mathrm{RBD}}({\mathit{Z}}_n;\tilde{\mathit{\beta }})\}=\mathbf{0}\},\hfill \end{array}$$

(11)

where ${\rho }_q(·,·)$ and ${\mathit{\psi }}_{\mathrm{RBD}}(·;·)$ are defined in Equations (4) and (7), respectively. It’s worth noting that the solution to ${\mathrm{E}}_K\{{\mathit{\psi }}_{\mathrm{RBD}}({\mathit{Z}}_n;\tilde{\mathit{\beta }})\}=\mathbf{0}$ may not be unique, i.e., ${\mathcal{S}}_K$ may contain more than one element. We then define a functional for the estimator of ${\tilde{\mathit{\beta }}}_{n,0}$ as follows,

$$\begin{array}{c}\hfill \mathit{T}(K)=\underset{\tilde{\mathit{\beta }}\in {\mathcal{S}}_K}{argmin}\parallel \tilde{\mathit{\beta }}-{\tilde{\mathit{\beta }}}_{n,0}\parallel .\end{array}$$

(12)

From the result of Lemma A1 in Appendix A, $\mathit{T}(K_{n,ϵ})$ is the unique local minimizer of ${\ell }_{K_{n,ϵ}}(\tilde{\mathit{\beta }})$ in the $\sqrt{p_n/n}$-neighborhood of ${\tilde{\mathit{\beta }}}_{n,0}$. Particularly, $\mathit{T}(K_{n,0})={\tilde{\mathit{\beta }}}_{n,0}$. Similarly, from Lemma A2 in Appendix A, $\mathit{T}({\mathbb{K}}_n)$ is the unique local minimizer of ${\ell }_{{\mathbb{K}}_n}(\tilde{\mathit{\beta }})$ which satisfies $\parallel \mathit{T}({\mathbb{K}}_n)-{\tilde{\mathit{\beta }}}_{n,0}\parallel =O_{\mathrm{P}}(\sqrt{p_n/n})$.

From [23] (Equation (2.1.6) on pp. 84), the influence function of $\mathit{T}(·)$ at $K_{n,0}$ is defined as

$$\begin{array}{c}\hfill \mathrm{IF}({\mathit{z}}_n;\mathit{T},K_{n,0})=\frac{\partial }{\partial t}\mathit{T}((1-t)K_{n,0}+t{\Delta }_{{\mathit{z}}_n}){|}_{t=0}=\underset{t\downarrow 0}{lim}\frac{\mathit{T}((1-t)K_{n,0}+t{\Delta }_{{\mathit{z}}_n})-{\tilde{\mathit{\beta }}}_{n,0}}{t},\end{array}$$

where ${\Delta }_{{\mathit{z}}_n}$ is the probability measure which puts mass 1 at the point ${\mathit{z}}_n$. Since the dimension of $\mathit{T}(·)$ diverges with n, its influence function is defined for each fixed n. From Lemma A8 in Appendix A, under certain regularity conditions, the influence function exists and has the following expression:

$$\begin{array}{c}\hfill \mathrm{IF}({\mathit{z}}_n;\mathit{T},K_{n,0})=-{\mathbf{H}}_n^{-1}{\mathit{\psi }}_{\mathrm{RBD}}({\mathit{z}}_n;{\tilde{\mathit{\beta }}}_{n,0}),\end{array}$$

(13)

where ${\mathbf{H}}_n={\mathrm{E}}_{K_{n,0}}\{{\mathrm{p}}_2(Y;{\tilde{\mathit{X}}}_n^T{\tilde{\mathit{\beta }}}_{n,0})w({\mathit{X}}_n){\tilde{\mathit{X}}}_n{\tilde{\mathit{X}}}_n^T\}$. The form of the influence function for diverging $p_n$ in Equation(13) coincides with that in [23,24] for fixed $p_n$.

In our theoretical derivations, approximations of the asymptotic level and power of $W_n$ will involve the following matrices:

$$\begin{array}{ccc}\hfill {\mathsf{\Omega }}_n& =& {\mathrm{E}}_{K_{n,0}}\{{\mathrm{p}}_1^2(Y;{\tilde{\mathit{X}}}_n^T{\tilde{\mathit{\beta }}}_{n,0})w^2({\mathit{X}}_n){\tilde{\mathit{X}}}_n{\tilde{\mathit{X}}}_n^T\},\hfill \\ \hfill U_n& =& A_n{\mathbf{H}}_n^{-1}{\mathsf{\Omega }}_n{\mathbf{H}}_n^{-1}{A}_n^T.\hfill \end{array}$$

3.1. Asymptotic Level of $W_n$ under Contamination

We now investigate the asymptotic level of the Wald-type test $W_n$ under the $ϵ$-contamination.

Theorem 1.

Assume Conditions A0–A9 and B4 in Appendix A. Suppose $p_n^6/n\to 0$ as $n\to \infty$, ${sup}_n{\mathrm{E}}_J(\parallel w({\mathit{X}}_n){\tilde{\mathit{X}}}_n\parallel )\le C$. Denote by $\alpha (K_{n,ϵ})$ the level of $W_n=n{\{A_n\mathit{T}({\mathbb{K}}_n)-{\mathit{g}}_0\}}^T{(A_n{\widehat{\mathbf{H}}}_n^{-1}{\widehat{\mathsf{\Omega }}}_n{\widehat{\mathbf{H}}}_n^{-1}{A}_n^T)}^{-1}\{A_n\mathit{T}({\mathbb{K}}_n)-{\mathit{g}}_0\}$ when the underlying distribution is $K_{n,ϵ}$ in Equation (10) and by ${\alpha }_{{}_0}$ the nominal level. Under $H_0$ in Equation (8), it follows that

$$\begin{array}{c}\hfill \underset{n\to \infty }{lim\; sup}\alpha (K_{n,ϵ})={\alpha }_{{}_0}+{ϵ}^2{\mu }_{k}D+o({ϵ}^2)\mathit{as}ϵ\to 0,\end{array}$$

where

$$\begin{array}{c}\hfill D=\underset{n\to \infty }{lim\; sup}{\parallel U_n^{-1/2}{A}_n{\mathrm{E}}_J\{\mathrm{IF}({\mathit{Z}}_n;\mathit{T},K_{n,0})\}\parallel }^2<\infty ,\end{array}$$

${\mu }_k=-\frac{\partial }{\partial \delta }{H}_k({\eta }_{{}_{1-{\alpha }_{{}_0}}};\delta ){|}_{\delta =0}$, $H_k(·;\delta )$ is the cumulative distribution function of a ${\chi }_k^2(\delta )$ distribution, and ${\eta }_{{}_{1-{\alpha }_{{}_0}}}$ is the $1-{\alpha }_{{}_0}$ quantile of the central ${\chi }_k^2$ distribution.

Theorem 1 indicates that if the influence function for $\mathit{T}(·)$ is bounded, then the asymptotic level of $W_n$ under the $ϵ$-contamination is also bounded and close to the nominal level when $ϵ$ is sufficiently small. As a comparison, the robustness property in [10] of the Wald-type test is studied based on M-estimator for general parametric models with a fixed dimension $p_n$. They assumed certain conditions that guarantee Fréchet differentiability which further implies the existence of the influence function and the asymptotic normality of the corresponding estimator. However, in the set-ups of our paper, it’s difficult to check those conditions, due to the use of Bregman divergence and the diverging dimension $p_n$. Hence, the assumptions we make in Theorem 1 are different from those in [10], and are comparatively mild and easy to check. Moreover, the result of Theorem 1 cannot be easily derived from that of [10].

In Theorem 1, $p_n$ is allowed to diverge with $p_n^6/n=o(1)$, which is slower than that in [1] with $p_n^5/n=o(1)$. Theoretically, the assumption $p_n^5/n=o(1)$ is required to obtain the asymptotic distribution of $W_n$ in [1]. Furthermore, to derive the limit distribution of $W_n$ under the $ϵ$-contamination, assumption $p_n^6/n=o(1)$ is needed (see Lemma A7 in Appendix A). Hence, the reason that our assumption is stronger than that in [1] is the consideration of the $ϵ$-contamination of the data. Practically, due to the advancement of technology and different forms of data gathering, large dimension becomes a common characteristic and hence the varying-dimensional model has a wide range of applications, e.g., brain imaging data, financial data, web term-document data and gene expression data. Even some of the classical settings, e.g., the Framingham heart study with $n=25,000$ and $p_n=100$, can be viewed as varying-dimensional cases.

As an illustration, we apply the general result of Theorem 1 to the special case of a point mass contamination.

Corollary 1.

With the notations in Theorem 1, assume Conditions $\mathrm{A}0$–$\mathrm{A}9$ in Appendix A, ${sup}_{{\mathit{x}}_n\in {\mathbb{R}}^{p_n}}\parallel w({\mathit{x}}_n){\mathit{x}}_n\parallel \le C$ and ${sup}_{\mu \in \mathbb{R}}|q^{″}(\mu )\sqrt{V(\mu )}/F^{\prime }(\mu )|\le C$.

(i)

If $p_n\equiv p$, $A_n\equiv A$, ${\tilde{\mathit{\beta }}}_{n,0}\equiv {\tilde{\mathit{\beta }}}_0$, $K_{n,0}\equiv K_0$ and $U_n\equiv U$ are fixed, then, for $K_{n,ϵ}=(1-ϵ/\sqrt{n})K_0+ϵ/\sqrt{n}{\Delta }_{\mathit{z}}$ with $\mathit{z}\in {\mathbb{R}}^p$ a fixed point, under $H_0$ in Equation (8), it follows that

$$\begin{array}{c}\hfill \underset{\mathit{z}\in {\mathbb{R}}^p}{sup}\underset{n\to \infty }{lim}\alpha (K_{n,ϵ})={\alpha }_{{}_0}+{ϵ}^2{\mu }_k{D}_1+o({ϵ}^2)\mathit{as}ϵ\to 0,\end{array}$$

where

$$\begin{array}{c}\hfill D_1=\underset{\mathit{z}\in {\mathbb{R}}^p}{sup}{\parallel U^{-1/2}A\mathrm{IF}(\mathit{z};\mathit{T},K_0)\parallel }^2<\infty .\end{array}$$

(ii)

If $p_n$ diverges with $p_n^6/n\to 0$, for $K_{n,ϵ}=(1-ϵ/\sqrt{n})K_{n,0}+ϵ/\sqrt{n}{\Delta }_{{\mathit{z}}_n}$ with ${\mathit{z}}_n\in {\mathbb{R}}^{p_n}$ a sequence of deterministic points, then, under $H_0$ in Equation (8),

$$\begin{array}{c}\hfill \underset{C_0>0}{sup}\underset{{\mathit{z}}_n\in S_{C_0}}{sup}\underset{n\to \infty }{lim\; sup}\alpha (K_{n,ϵ})={\alpha }_{{}_0}+{ϵ}^2{\mu }_k{D}_2+o({ϵ}^2)\mathit{as}ϵ\to 0,\end{array}$$

where $S_{C_0}=\{{\mathit{z}}_n={({\mathit{x}}_n^T,y)}^T:\parallel {\mathit{x}}_n{\parallel }_{\infty }\le C_0\}$, $C_0>0$ is a constant and

$$\begin{array}{c}\hfill D_2=\underset{C_0>0}{sup}\underset{{\mathit{z}}_n\in S_{C_0}}{sup}\underset{n\to \infty }{lim\; sup}{\parallel U_n^{-1/2}{A}_n\mathrm{IF}({\mathit{z}}_n;\mathit{T},K_{n,0})\parallel }^2<\infty .\end{array}$$

In Corollary 1, conditions ${sup}_{{\mathit{x}}_n\in {\mathbb{R}}^{p_n}}\parallel w({\mathit{x}}_n){\mathit{x}}_n\parallel \le C$ and ${sup}_{\mu \in \mathbb{R}}|q^{″}(\mu )\sqrt{V(\mu )}/F^{\prime }(\mu )|\le C$ are needed to guarantee the boundedness of the score function in Equation (7). Particularly, the function $w({\mathit{x}}_n)$ downweights the high leverage points and can be chosen as, e.g., $w({\mathit{x}}_n)=1/(1+\parallel {\mathit{x}}_n\parallel )$. The condition ${sup}_{\mu \in \mathbb{R}}|q^{″}(\mu )\sqrt{V(\mu )}/F^{\prime }(\mu )|\le C$ is needed to bound Equation (6), and is satisfied in many situations.

• For example, for the linear model with $q(\mu )=a\mu -{\mu }^2$, $V(\mu )={\sigma }^2$ and $F(\mu )=\mu$, where a and ${\sigma }^2$ are constants, we observe $|q^{″}(\mu )\sqrt{V(\mu )}/F^{\prime }(\mu )|=2\sigma \le C$.
• Another example is the logistic regression model with binary response and $q(\mu )=-2\{\mu log(\mu )+(1-\mu )log(1-\mu )\}$ (corresponding to Bernoulli deviance loss), $V(\mu )=\mu (1-\mu )$, $F(\mu )=log\{\mu /(1-\mu )\}$. In this case, $|q^{″}(\mu )\sqrt{V(\mu )}/F^{\prime }{(\mu )|=2\{\mu (1-\mu )\}}^{1/2}\le C$ since $\mu \in [0,1]$. Likewise, if $q(\mu )=2{\{\mu (1-\mu )\}}^{1/2}$ (for the exponential loss), then $|q^{″}(\mu )\sqrt{V(\mu )}/F^{\prime }(\mu )|=1/2$.

Furthermore, the bound on $\psi (·)$ is useful to control deviations in the Y-space, which ensures the stability of the robust-$\mathrm{BD}$ test if Y is arbitrarily contaminated.

Concerning the dimensionality $p_n$, Corollary 1 reveals the following implications. If $p_n$ is fixed, then the asymptotic level of $W_n$ under the $ϵ$-contamination is uniformly bounded for all $\mathit{z}\in {\mathbb{R}}^p$, which implies the robustness of validity of the test. This result coincides with that in Proposition 5 of [10]. When $p_n$ diverges, the asymptotic level is still stable if the point contamination satisfies $\parallel {\mathit{x}}_n{\parallel }_{\infty }\le C_0$, where $C_0>0$ is an arbitrary constant. Although this condition may not be the weakest, it still covers a wide range of point mass contaminations.

3.2. Asymptotic Power of $W_n$ under Contamination

Now, we will study the asymptotic power of $W_n$ under a sequence of contiguous alternatives of the form

$$\begin{array}{c}\hfill H_{1n}:A_n{\tilde{\mathit{\beta }}}_{n,0}-{\mathit{g}}_0=n^{-1/2}\mathit{c},\end{array}$$

(14)

where $\mathit{c}={(c_1,\dots ,c_{\mathrm{k}})}^T\ne \mathbf{0}$ is fixed.

Theorem 2.

Assume Conditions $\mathrm{A}0$–$\mathrm{A}9$ and $\mathrm{B}4$ in Appendix A. Suppose $p_n^6/n\to 0$ as $n\to \infty$, ${sup}_n{\mathrm{E}}_J(\parallel w({\mathit{X}}_n){\tilde{\mathit{X}}}_n\parallel )\le C$. Denote by $\beta (K_{n,ϵ})$ the power of $W_n=n{\{A_n\mathit{T}({\mathbb{K}}_n)-{\mathit{g}}_0\}}^T{(A_n{\widehat{\mathbf{H}}}_n^{-1}{\widehat{\mathsf{\Omega }}}_n{\widehat{\mathbf{H}}}_n^{-1}{A}_n^T)}^{-1}\{A_n\mathit{T}({\mathbb{K}}_n)-{\mathit{g}}_0\}$ when the underlying distribution is $K_{n,ϵ}$ in Equation (10) and by ${\beta }_0$ the nominal power. Under $H_{1n}$ in Equation (14), it follows that

$$\begin{array}{c}\hfill \underset{n\to \infty }{lim\; inf}\beta (K_{n,ϵ})={\beta }_0+ϵ{\nu }_{k}B+o(ϵ)\mathit{as}ϵ\to 0,\end{array}$$

where

$$\begin{array}{c}\hfill B=\underset{n\to \infty }{lim\; inf}2{\mathit{c}}^T{U}_n^{-1}{A}_n{\mathrm{E}}_J\{\mathrm{IF}({\mathit{Z}}_n;\mathit{T},K_{n,0})\},\end{array}$$

with $\left|B\right|<\infty$, ${\nu }_k=-\frac{\partial }{\partial \delta }{H}_k({\eta }_{{}_{1-{\alpha }_{{}_0}}};\delta ){|}_{\delta ={\mathit{c}}^T{U}_n^{-1}\mathit{c}}$ and $H_k(·;\delta )$ and ${\eta }_{{}_{1-{\alpha }_{{}_0}}}$ being defined in Theorem 1.

The result for the asymptotic power is similar in spirit to that for the level. From Theorem 2, if the influence function is bounded, the asymptotic power is also bounded from below and close to the nominal power under a small amount of contamination. This means that the robust-$\mathrm{BD}$ Wald-type test enjoys the robustness of efficiency. In addition, the property of the asymptotic power can be obtained for a point mass contamination.

Corollary 2.

With the notations in Theorem 2, assume Conditions $\mathrm{A}0$–$\mathrm{A}9$ in Appendix A, ${sup}_{{\mathit{x}}_n\in {\mathbb{R}}^{p_n}}\parallel w({\mathit{x}}_n){\mathit{x}}_n\parallel \le C$ and ${sup}_{\mu \in \mathbb{R}}|q^{″}(\mu )\sqrt{V(\mu )}/F^{\prime }(\mu )|\le C$.

(i)

If $p_n\equiv p$, $A_n\equiv A$, ${\tilde{\mathit{\beta }}}_{n,0}\equiv {\tilde{\mathit{\beta }}}_0$, $K_{n,0}\equiv K_0$ and $U_n\equiv U$ are fixed, then, for $K_{n,ϵ}=(1-ϵ/\sqrt{n})K_0+ϵ/\sqrt{n}{\Delta }_{\mathit{z}}$ with $\mathit{z}\in {\mathbb{R}}^p$ a fixed point, under $H_{1n}$ in Equation (14), it follows that

$$\begin{array}{c}\hfill \underset{\mathit{z}\in {\mathbb{R}}^p}{inf}\underset{n\to \infty }{lim}\beta (K_{n,ϵ})={\beta }_0+ϵ{\nu }_k{B}_1+o(ϵ)\mathit{as}ϵ\to 0,\end{array}$$

where

$$\begin{array}{c}\hfill B_1=\underset{\mathit{z}\in {\mathbb{R}}^p}{inf}2{\mathit{c}}^T{U}^{-1}A\mathrm{IF}(\mathit{z};\mathit{T},K_0),\end{array}$$

with $|B_1|<\infty$.

(ii)

If $p_n$ diverges with $p_n^6/n\to 0$, for $K_{n,ϵ}=(1-ϵ/\sqrt{n})K_{n,0}+ϵ/\sqrt{n}{\Delta }_{{\mathit{z}}_n}$ with ${\mathit{z}}_n\in {\mathbb{R}}^{p_n}$ a sequence of deterministic points, then, under $H_{1n}$ in Equation (14),

$$\begin{array}{c}\hfill \underset{C_0>0}{inf}\underset{{\mathit{z}}_n\in S_{C_0}}{inf}\underset{n\to \infty }{lim\; inf}\beta (K_{n,ϵ})={\beta }_0+ϵ{\nu }_k{B}_2+o(ϵ)\mathit{as}ϵ\to 0,\end{array}$$

where $S_{C_0}=\{{\mathit{z}}_n={({\mathit{x}}_n^T,y)}^T:\parallel {\mathit{x}}_n{\parallel }_{\infty }\le C_0\}$, $C_0>0$ is a constant and

$$\begin{array}{c}\hfill B_2=\underset{C_0>0}{inf}\underset{{\mathit{z}}_n\in S_{C_0}}{inf}\underset{n\to \infty }{lim\; inf}2{\mathit{c}}^T{U}_n^{-1}{A}_n\mathrm{IF}({\mathit{Z}}_n;\mathit{T},K_{n,0}),\end{array}$$

with $|B_2|<\infty$.

4. Simulation

Regarding the practical utility of $W_n$, numerical studies concerning the empirical level and power of $W_n$ under a fixed amount of contamination have been conducted in Section 6 of [1]. To support the theoretical results in our paper, we conduct new simulations to check the robustness of validity and efficiency of $W_n$. Specifically, we will examine the empirical level and power of the test statistic as $ϵ$ varies.

The robust-$\mathrm{BD}$ estimation utilizes the Huber $\psi$-function ${\psi }_c(·)$ with $c=1.345$ and the weight function $w({\mathit{X}}_n)=1/(1+\parallel {\mathit{X}}_n\parallel )$. Comparisons are made with the classical non-robust counterparts corresponding to using $\psi (r)=r$ and $w({\mathit{x}}_n)\equiv 1$. For each situation below, we set $n=1000$ and conduct 400 replications.

4.1. Overdispersed Poisson Responses

Overdispersed Poisson counts Y, satisfying $\mathrm{var}(Y|{\mathit{X}}_n={\mathit{x}}_n)=2m({\mathit{x}}_n)$, are generated via a negative Binomial $(m({\mathit{x}}_n),1/2)$ distribution. Let $p_n=\lfloor 4(n^{1/5.5}-1)\rfloor$ and ${\tilde{\mathit{\beta }}}_{n,0}={(0,2,0,\dots ,0)}^T$, where $\lfloor ·\rfloor$ denotes the floor function. Generate ${\mathit{X}}_{ni}={(X_{i,1},\dots ,X_{i,p_n})}^T$ by $X_{i,j}\stackrel{\mathrm{i}.\mathrm{i}.\mathrm{d}.}{\sim }\mathrm{Unif}[-0.5,0.5]$. The log link function is considered and the (negative) quasi-likelihood is utilized as the BD, generated by the q-function in Equation (3) with $V(\mu )=\mu$. The estimator and test statistic are calculated by assuming Y follows Poisson distribution.

The data are contaminated by $X_{i,\mathrm{mod}(i,p_n-1)+1}^{\ast }=3\mathrm{sign}(U_i-0.5)$ and $Y_i^{\ast }=Y_i\mathrm{I}(Y_i>20)+20\mathrm{I}(Y_i\le 20)$ for $i=1,\dots ,k$, with $k\in \{2,4,6,8,10,12,14,16\}$ the number of contaminated data points, where $\mathrm{mod}(a,b)$ is the modulo operation “a modulo b” and $\{U_i\}\stackrel{\mathrm{i}.\mathrm{i}.\mathrm{d}.}{\sim }\mathrm{Unif}(0,1)$. Then, the proportion of contaminated data, $k/n$, is equal to $ϵ/\sqrt{n}$ as in Equation (10), which implies $ϵ=k/\sqrt{n}$.

Consider the null hypothesis $H_0:A_n{\tilde{\mathit{\beta }}}_{n,0}=0$ with $A_n=(0,0,0,1,0,\dots ,0)$. Figure 1 plots the empirical level of $W_n$ versus $ϵ$. We observe that the asymptotic nominal level 0.05 is approximately retained by the robust Wald-type test. On the other hand, under contaminations, the non-robust Wald-type test breaks in level, showing high sensitivity to the presence of outliers.

Figure 1. Observed level of $W_n$ versus $ϵ$ for overdispersed Poisson responses. The dotted line indicates the 5% significance level.

To assess the stability of the power of the test, we generate the original data from the true model, but with the true parameter ${\tilde{\mathit{\beta }}}_{n,0}$ replaced by ${\tilde{\mathit{\beta }}}_n={\tilde{\mathit{\beta }}}_{n,0}+\delta \mathit{c}$ with $\delta \in \{-0.4,0.4,-0.6,0.6\}$ and $\mathit{c}={(1,\dots ,1)}^T$ a vector of ones. Figure 2 plots the empirical rejection rates of the null model, which implies that the robust Wald-type test has sufficiently large power to detect the alternative hypothesis. In addition, the power of the robust method is generally larger than that of the non-robust method.

Figure 2. Observed power of $W_n$ versus $ϵ$ for overdispersed Poisson responses. The statistics in the left panel correspond to non-robust method and those in the right panel are for robust method. The asterisk line indicates the 5% significance level.

4.2. Bernoulli Responses

We generate data with two classes from the model, $Y|{\mathit{X}}_n={\mathit{x}}_n\sim \mathrm{Bernoulli}\{m({\mathit{x}}_n)\}$, where $\mathrm{logit}\{m({\mathit{x}}_n)\}={\tilde{\mathit{x}}}_n^T{\tilde{\mathit{\beta }}}_{n,0}$. Let $p_n=2$, ${\tilde{\mathit{\beta }}}_{n,0}={(0,1,1)}^T$ and ${\mathit{X}}_{ni}\stackrel{\mathrm{i}.\mathrm{i}.\mathrm{d}.}{\sim }N(\mathbf{0},{\mathbf{I}}_{p_n})$. The null hypothesis is $H_0:{\tilde{\mathit{\beta }}}_{n,0}={(0,1,1)}^T$. Both the deviance loss and the exponential loss are employed as the BD. We contaminate the data by setting $X_{i,1}^{\ast }=2+i/8$ and $Y_i^{\ast }=0$ for $i=1,\dots ,k$ with $k\in \{2,4,6,8,10,12,14,16\}$. To investigate the robustness of validity of $W_n$, we plot the observed level versus $ϵ$ in Figure 3. We find that the level of the non-robust method diverges fast as $ϵ$ increases. It’s also clear that the empirical level of the robust method is close to the nominal level when $ϵ$ is small and increases slightly with $ϵ$, which coincides with our results in Theorem 1.

Figure 3. Observed level of $W_n$ versus $ϵ$ for Bernoulli responses. The statistics in (a) use deviance loss and those in (b) use exponential loss. The dotted line indicates the 5% significancelevel.

To assess the stability of the power of $W_n$, we generate the original data from the true model, but with the true parameter ${\tilde{\mathit{\beta }}}_{n,0}$ replaced by ${\tilde{\mathit{\beta }}}_n={\tilde{\mathit{\beta }}}_{n,0}+\delta \mathit{c}$ with $\delta \in \{-0.1,0.2,-0.3,0.4\}$ and $\mathit{c}={(1,\dots ,1)}^T$ a vector of ones. Figure 4 plots the power of the Wald-type test versus $ϵ$, which implies that the robust method has sufficiently large power, and hence supports the theoretical results in Theorem 2.

Figure 4. Observed power of $W_n$ versus $ϵ$ for Bernoulli responses. The top panels correspond to deviance loss while the bottom panels are for exponential loss. The statistics in the left panels are calculated using non-robust method and those in the right panels are from robust method. The asterisk line indicates the 5% significance level.