Generalized Information Matrix Tests for Detecting Model Misspecification

Abstract

Generalized Information Matrix Tests (GIMTs) have recently been used for detecting the presence of misspecification in regression models in both randomized controlled trials and observational studies. In this paper, a unified GIMT framework is developed for the purpose of identifying, classifying, and deriving novel model misspecification tests for finite-dimensional smooth probability models. These GIMTs include previously published as well as newly developed information matrix tests. To illustrate the application of the GIMT framework, we derived and assessed the performance of new GIMTs for binary logistic regression. Although all GIMTs exhibited good level and power performance for the larger sample sizes, GIMT statistics with fewer degrees of freedom and derived using log-likelihood third derivatives exhibited improved level and power performance.

1. Introduction

If a researcher’s probability model of the observed data is not correctly specified, then the interpretation of its parameter estimates may not be valid, leading to incomplete or incorrect conclusions. Thus, whether a model is correctly specified must be considered when analyzing and interpreting data (e.g., [1,2]). This issue is critically important in econometrics as well as more general scientific inquiry. For example, in health economics, estimates of the impact of clinical treatments [3,4], care systems [5], and health policy interventions on health outcomes [6] are dependent on the underlying assumption that the model to be tested is correctly specified. Further, model misspecification testing is essential for statistical analysis of randomized control trials [7,8] and observational studies [9,10]. For these reasons, this paper introduces a unified framework for identifying, classifying, and developing a wide range of specification tests.

1.1. Information Matrix Test Methods for Detection of Model Misspecification

Assume that the data $x_1,\mathrm{...},x_n$ observed in an experiment is a realization of a sequence of independent and identically distributed d-dimensional random vectors $X_1,\mathrm{...},X_n$ with a common data generating process density $p_x$. Let $\mathcal{M}\equiv \left\{f\left(x;\mathsf{\theta }\right):\mathsf{\theta }\in \mathsf{\Theta }\right\}$ denote a proposed probability model that is a collection of probability densities indexed by a k-dimensional parameter vector $\mathsf{\theta }$. If $p_x\in \mathcal{M}$, so that $p_x\left(x\right)=f\left(x;{\mathsf{\theta }}^{\ast }\right)$ $a.e.$ for some ${\mathsf{\theta }}^{\ast }\in \mathsf{\Theta }$, then $\mathcal{M}$ is correctly specified with respect to $p_x$.

When $\mathcal{M}$ is correctly specified with respect to $p_x$, the inverse of the asymptotic covariance matrix of the maximum likelihood estimator ${\widehat{\mathsf{\theta }}}_n\equiv \arg \max {\displaystyle \prod _{i=1}^{n}f\left(X_i;\mathsf{\theta }\right)}$ is equal to both the inverse Hessian covariance matrix $A^{\ast }\equiv -{\mathsf{\nabla }}^{2}E\left\{\log f\left(X_i;{\mathsf{\theta }}^{\ast }\right)\right\}$ and the inverse Outer Product Gradient (OPG) covariance matrix $B^{\ast }\equiv E\left\{\mathsf{\nabla }\log f\left(X_i;{\mathsf{\theta }}^{\ast }\right){\left(\mathsf{\nabla }\log f\left(X_i;{\mathsf{\theta }}^{*}\right)\right)}^T\right\}$. This classic result is called the Information Matrix Equality (see [1,2], and Theorem 4 of this paper for relevant reviews).

Let $u:{\mathcal{R}}^k\to \mathcal{R}$. The notation $\mathsf{\nabla }u$ refers to a k-dimensional column vector of functions called the gradient whose ith element is $\frac{\partial u}{\partial x_i}$, $i=1,\mathrm{...},k.$ The notation ${\mathsf{\nabla }}^{2}u$ refers to a k-dimensional matrix-valued function which is called the Hessian of u. The element in the ith row and jth column of ${\mathsf{\nabla }}^{2}u$ is $\frac{{\partial }^{2}u}{\partial x_i\partial x_j}$, $i,j=1,\mathrm{...},k.$

As described by White [1,2], the information matrix equality may be used as the basis for a test of model misspecification. White [1] proposed the Information Matrix Test (IMT) for testing the null hypothesis that the elements of the k-dimensional Hessian and k-dimensional Outer Product Gradient (OPG) inverse asymptotic covariance matrices (denoted by $A^{\ast }$ and $B^{\ast }$ respectively) are equal. That is, White [1] considered the null hypothesis: $H_o:vech\left(A^{\ast }-B^{\ast }\right)=0_{k(k+1)/2}$ where $0_{k(k+1)/2}$ denotes a k(k + 1)/2-dimensional column vector of zeros. Rejection of this null hypothesis thus implies a violation of the information matrix equality and thus the presence of model misspecification. Moreover, as noted by White [1], it may be helpful to also consider situations where the null hypothesis is “directional.” If a directional null hypothesis is rejected, this implies $H_o:vech\left(A^{\ast }-B^{\ast }\right)=0_{k(k+1)/2}$ is rejected (but the converse of this latter statement does not hold). White [1], in particular, discussed directional IMTs that have the form: $H_o:Svech\left(A^{\ast }-B^{\ast }\right)=0_r$ where the selection matrix $S\in {\mathcal{R}}^{r\times (k(k+1)/2)}$ consists of r rows of a k(k + 1)/2-dimensional identity matrix. In some cases directional IMTs may have more statistical power because they are designed to identify specific types of model misspecification.

For many years, the IMT approach has not been widely used outside of linear regression modeling because various instabilities (possibly associated with large degrees of freedom) of the test were observed. Chesher [11] and Lancaster [12] demonstrated how the calculation of the third derivatives of the log-likelihood function could be avoided for the full IMT, but the effectiveness of their approach was shown in some cases to exhibit unacceptable performance in logistic regression and linear regression [13,14,15,16,17,18].

1.2. Recent Developments in Information Matrix Test Theory

An advance in the theory of information matrix testing was provided by Presnell and Boos [19] (also see, [20,21,22]), who introduced an IOS (in and out of sample) directional IMT and showed that it was effective in a variety of important situations through both theoretical analyses and simulation studies. More recently, Golden et al. [23] introduced a general unified theory for model specification testing based upon a nonlinear extension of White’s [1] approach to specification testing. The new IMTs developed within the framework of Golden et al. [23] are called Generalized Information Matrix Tests (GIMT).

In particular, Golden et al. [23] discussed the problem of testing the null hypothesis that a smooth nonlinear GIMT hypothesis function $s:{\mathcal{R}}^{k\times k}\times {\mathcal{R}}^{k\times k}\to {\mathcal{R}}^r$ of the Hessian and OPG inverse asymptotic covariance matrices is equal to an r-dimensional vector of zeros. That is, a GIMT tests the null hypothesis $H_o:s\left(A^{\ast },B^{\ast }\right)=0_r$. Golden et al. [23] emphasized that different choices of GIMT hypothesis function yield different types of directional and non-directional GIMT hypotheses. Although Golden et al. [23] did not provide explicit regularity conditions and a detailed analysis of their proposed general class of GIMTs, Golden et al. [23] introduced key formal definitions, provided an informal discussion of relevant theoretical results, and reported the results of a comprehensive simulation study of a realistic epidemiological analysis problem using logistic regression for six new GIMTs that exhibited appealing level and power performance. This approach for the detection of model misspecification has now been used in observational and randomized controlled trial studies [7,8,9,10].

Since the publication of Golden et al. [23], Cho and White [24] described an important class of non-directional GIMTs and showed that each of their three test statistics for model misspecification was asymptotically distributed as a squared Gaussian random variable under the null hypothesis. In addition, Cho and White [24] provided analyses of the power of their test statistics under local and global alternatives. Zhou et al. [25] proposed a non-directional GIMT statistic for the large important class of regression models where the distribution of the response variable conditioned upon the covariates is a member of the linear exponential family. Like Cho and White [24], they showed their misspecification test statistic has only a single degree of freedom and is asymptotically distributed as a squared Gaussian random variable under the null hypothesis. Huang and Prokhorov [26] also showed how the information matrix testing framework is useful for investigating goodness-of-fit using non-directional GIMT statistics for semi-parametric probability models that are specified by copulas. All of this previous work on GIMTs can be interpreted as special cases or variants of special cases of the general framework of Golden et al. [23] for finite-dimensional smooth probability models.

This paper provides a unified framework for addressing the detection of model misspecification using a variety of GIMT statistics for a large class of finite-dimensional smooth probability models. By presenting the details of the GIMT framework and explicitly presenting the relevant regularity assumptions, it establishes the foundation for supporting research into the further development of a large class of GIMTs as well as assisting in understanding the similarities and differences between different GIMTs in the existing published statistical literature.

Our paper is organized in the following manner. In Section 2, we provide the assumptions of the GIMT framework. In Section 3, we characterize the asymptotic distribution of a large family of GIMTs for a large class of finite-dimensional smooth probability models under the assumptions and definitions in Section 2. In Section 4, we investigate the performance of new GIMTs using simulation studies developed with respect to a particular logistic regression model intended to be representative of a commonly encountered problem of model misspecification detection. Conclusions are provided in Section 5.

2. GIMT Theoretical Framework: Definitions and Assumptions

In this section, we introduce the definitions and assumptions of our formal mathematical theory of Generalized Information Matrix Tests. In most practical applications, these assumptions are often satisfied for thrice continuously differentiable probability models with a fixed number of free parameters that have locally unique solutions. Throughout, it is assumed that observations are independent and identically distributed.

2.1. Data Generating Process

Let $\mathcal{B}\left({\mathcal{R}}^d\right)$ be the Borel σ-field generated by the open subsets of ${\mathcal{R}}^d$.

Assumption 1.

Data Generating Process (DGP). Let $X_i,i=1,2,\mathrm{...}$ be a sequence of independent and identically distributed (i.i.d) random vectors where $X_i$ has a common probability measure P on the measurable space $\left({\mathcal{R}}^d,\mathcal{B}\left({\mathcal{R}}^d\right)\right)$ with completion $\left({\mathcal{R}}^d,{\mathcal{F}}_0,P_0\right)$.

Let the triplet $(\mathrm{\Omega },{\mathcal{F}}_o,{\mathcal{P}}_o)$ be the probability space for the Data Generating Process (DGP).

In regression modeling applications, the first element of the d-dimensional real vector x_i (a realization of X_i) may be a particular value of the outcome (dependent) variable for a regression model associated with the ith data record, the second element of x_i may be the number 1 for the purpose of introducing an intercept parameter, and the remaining elements of x_i may be particular values for the predictor variables associated with the ith data record, i = 1, …, n.

Although Assumption 1 assumes that the observed data X_i, i = 1, 2, … are i.i.d., the theory presented here is also applicable to panel data analyses. For example, consider a situation where data are collected in a longitudinal study on a group of individuals over a period of time. Assume the observations across participants are assumed to be i.i.d., but the observations for a particular participant are neither necessarily identically distributed nor independent. Let X_it denote the observation associated with the measurement of the ith participant in the study at time index t for t = 1, … ,T (where T is a fixed finite number) and i = 1, …, n. The theory described in this article is applicable to evaluating the degree to which a probability model can account for the observed data ${\mathit{X}}_i\equiv \left[\begin{array}{ccc}{\mathit{X}}_{i,1}& \dots & {\mathit{X}}_{i,T}\end{array}\right]$, i = 1, …, n.

The following assumption of absolute continuity is now introduced to permit alternative representations of $P_0$ in order to represent, construct, and manipulate probability densities for data generating processes involving data samples containing combinations of discrete and continuous random variables.

Assumption 2.

Absolute Continuity. Let ${\nu }_j\left(x_j\right)$ be a σ-finite measure on the measurable space $\left(\mathcal{R},\mathcal{B}\left(\mathcal{R}\right)\right)$ , j = 1, … , d. Let $\nu \equiv \underset{j=1}{\overset{d}{{\displaystyle \otimes }}}{\nu }_j\left(x_j\right)$ be a σ-finite product measure on the measurable space $\left({\mathcal{R}}^d,\mathcal{B}\left({\mathcal{R}}^d\right)\right)$. Assume $P_0$ is absolutely continuous with respect to $\nu$.

By the Radon-Nikodým Theorem, Assumption 2 guarantees the joint distribution of X_i, P₀, may be represented using a Radon-Nikodým density function. The Radon-Nikodým density $p_x\equiv d{P}_0/d\nu$ is common to the i.i.d. random variables $X_i$, i = 1, …, n on the measurable space $\left({\mathcal{R}}^d,\mathcal{B}\left({\mathcal{R}}^d\right)\right)$.

Assumption 2 allows the theoretical results developed here to be applicable to random vectors that contain both discrete and absolutely continuous components. If a random vector is a discrete random vector or an absolutely continuous random vector, then the Radon-Nikodým density becomes a probability mass function or an absolutely continuous probability density function and the associated measure theory notation may be avoided.

2.2. Probability Model

Let $supp\mathit{X}$ denote the support of X.

Assumption 3.

Parametric Densities. (i) Let $\mathsf{\Theta }$ be a compact and non-empty subset of ${\mathcal{R}}^k$, $k\in \mathbb{N}$; (ii) Let $f:{\mathcal{R}}^d\times \mathsf{\Theta }\to [0,\infty )$. For each θ in $\mathsf{\Theta }$, $f(\cdot ;\mathsf{\theta })$ is a density with respect to v and $f(x;\cdot )$ is continuous on $\mathsf{\Theta }$ for each $\mathit{x}\in supp\mathit{X}$; (iii) $\log f(x;\cdot )$ is continuously differentiable on $\mathsf{\Theta }$ for each $\mathit{x}\in supp\mathit{X}$; (iv) $\log f(x;\cdot )$ is twice continuously differentiable on $\mathsf{\Theta }$ for each $\mathit{x}\in supp\mathit{X}$; (v) $\log f(x;\cdot )$ is thrice continuously differentiable on $\mathsf{\Theta }$ for each $\mathit{x}\in supp\mathit{X}$.

Definition. 

Probability Model. Let $f$ be defined as in Assumption 3(i) and Assumption 3(ii). Let $F:{\mathcal{R}}^d\times \mathsf{\Theta }\to [0,1]$ be defined such that for each θ in $\mathsf{\Theta }$, $F\left(\cdot ;\mathsf{\theta }\right):{\mathcal{R}}^d\to [0,1]$ is the probability distribution for X specified by density $f\left(\cdot ;\mathsf{\theta }\right).$ The set $\mathcal{M}\equiv \left\{F\left(\cdot ;\mathsf{\theta }\right):{\mathcal{R}}^d\to [0,1]|\mathsf{\theta }\in \mathsf{\Theta }\right\}$ is the probability model on $\mathbf{\Theta }$ specified by $f$.

Definition. 

Misspecified Model. The probability model $\mathcal{M}$ is misspecified when $P_0\notin \mathcal{M}$, otherwise $\mathcal{M}$ is correctly specified.

2.3. Hypothesis Function

Definition. 

GIMT Hypothesis Function. Let $\mathsf{\Upsilon }$ be a compact and non-empty subset of ${\mathcal{R}}^{k\times k}$, $k\in \mathbb{N}$. A Generalized Information Matrix Test (GIMT) Hypothesis function $s:\mathsf{\Upsilon }\times \mathsf{\Upsilon }\to {\mathcal{R}}^r$ has the property that if $A=B$, then $s\left(A,B\right)=0_r$ for every symmetric positive definite matrix $A\in \mathsf{\Upsilon }$ and for every symmetric positive definite matrix $B\in \mathsf{\Upsilon }$.

Definition. 

Nondirectional and directional GIMT Hypothesis Functions. Let $\mathsf{\Upsilon }$ be a compact and non-empty subset of ${\mathcal{R}}^{k\times k}$, $k\in \mathbb{N}$. A nondirectional GIMT hypothesis function $s:\mathsf{\Upsilon }\times \mathsf{\Upsilon }\to {\mathcal{R}}^r$ has the property $A=B$ if and only if $s\left(A,B\right)=0_r$ for all $\left(A,B\right)\in \mathsf{\Upsilon }\times \mathsf{\Upsilon }$. A directional GIMT hypothesis function is a GIMT hypothesis function that is not nondirectional.

When $A:{\mathcal{R}}^{m\times n}\to {\mathcal{R}}^{q\times r}$, let $\frac{dA}{dB}\equiv \frac{dvec\left(A^T\right)}{dvec\left(B^T\right)}$ when it exists (e.g., [27]; also see [28,29]). Let $\mathsf{\nabla }s:\mathsf{\Upsilon }\times \mathsf{\Upsilon }\to {\mathcal{R}}^{r\times 2k^2}$ be defined such that for all $A,B\in \mathsf{\Upsilon }$: $\mathsf{\nabla }s\left(A,B\right)\equiv \left[\begin{array}{cc}\frac{\partial s\left(\cdot ,B\right)}{\partial vec\left(A\right)}& \frac{\partial s\left(A,\cdot \right)}{\partial vec\left(B\right)}\end{array}\right]$ when it exists.

Assumption 4.

Hypothesis Function Regularity Conditions. (i) Let $\mathsf{\Upsilon }$ be a compact and non-empty subset of ${\mathcal{R}}^{k\times k}$, $k\in \mathbb{N}$. Let $s:\mathsf{\Upsilon }\times \mathsf{\Upsilon }\to {\mathcal{R}}^r$ be continuous on $\mathsf{\Upsilon }\times \mathsf{\Upsilon }$; (ii) $A^{\ast }$ and $B^{\ast }$ are in the interior of $\mathsf{\Upsilon }\subseteq {\mathcal{R}}^{k\times k}$; (iii) $\mathsf{\nabla }s$ exists and is continuous on $\mathsf{\Upsilon }\times \mathsf{\Upsilon }$; (iv) $\mathsf{\nabla }{s}^{\ast }$ has full row rank r on $\mathsf{\Upsilon }\times \mathsf{\Upsilon }$.

In practice, Assumption 4 provides a procedure for checking if the theory described here can be applied to a proposed GIMT hypothesis function.

Definition. 

Antisymmetric GIMT Hypothesis Function. Let $s:\mathsf{\Upsilon }\times \mathsf{\Upsilon }\to {\mathcal{R}}^r$ be a GIMT hypothesis function satisfying Assumption 4(i), Assumption 4(ii), and Assumption 4(iii). If, in addition, $s\left(A,B\right)=-s\left(B,A\right)$ for all $\left(A,B\right)\in \mathsf{\Upsilon }\times \mathsf{\Upsilon }$, then $s:\mathsf{\Upsilon }\times \mathsf{\Upsilon }\to {\mathcal{R}}^r$ is called an antisymmetric GIMT hypothesis function.

2.4. Notation

Let $g\left(\mathbf{x};\mathsf{\theta }\right)\equiv -\mathsf{\nabla }\log f\left(\mathbf{x};\mathsf{\theta }\right)$. Let ${\overline{g}}_n\left(\mathsf{\theta }\right)\equiv (1/n){\displaystyle \sum _{i=1}^{n}g\left({\mathbf{X}}_i;\mathsf{\theta }\right)}$. Let ${\widehat{g}}_n\equiv {\overline{g}}_n\left({\widehat{\mathsf{\theta }}}_n\right)$.

Let ${\overline{A}}_n\left(\mathsf{\theta }\right)\equiv -(1/n){\displaystyle \sum _{i=1}^n{\mathsf{\nabla }}^2\log f\left({\mathbf{X}}_i;\mathsf{\theta }\right)}$. Let $A\left(\mathsf{\theta }\right)\equiv {\mathsf{\nabla }}^{2}l\left(\mathsf{\theta }\right)$.

Let ${\overline{B}}_n\left(\mathsf{\theta }\right)\equiv (1/n){\displaystyle \sum _{i=1}^{n}g\left({\mathbf{X}}_i;\mathsf{\theta }\right)}{\left(g\left({\mathbf{X}}_i;\mathsf{\theta }\right)\right)}^T$.

Let $B\left(\mathsf{\theta }\right)\equiv {\displaystyle \int g\left(x;\mathsf{\theta }\right){\left(g\left(x;\mathsf{\theta }\right)\right)}^T}{p}_x\left(x\right)d{\nu }_x\left(x\right)$. Let ${\widehat{A}}_n\equiv {\overline{A}}_n\left({\widehat{\mathsf{\theta }}}_n\right)$. Let ${\widehat{B}}_n\equiv {\overline{B}}_n\left({\widehat{\mathsf{\theta }}}_n\right)$.

Let $A^{\ast }=A\left({\mathsf{\theta }}^{\ast }\right)$. Let $B^{\ast }=B\left({\mathsf{\theta }}^{\ast }\right)$. Let $d_{x,\mathsf{\theta }}\left(x;\mathsf{\theta }\right)\equiv \left[\begin{array}{c}-vech\left({\mathsf{\nabla }}^2\log f\left(\mathbf{x};\mathsf{\theta }\right)\right)\\ vech\left(g\left(x;\mathsf{\theta }\right){\left(g\left(x;\mathsf{\theta }\right)\right)}^T\right)\end{array}\right]$.

Let ${\overline{d}}_n\left(\mathsf{\theta }\right)=(1/n){\displaystyle \sum _{i=1}^n{d}_{x,\mathsf{\theta }}\left(X_i;\mathsf{\theta }\right)}$. Let ${\widehat{d}}_n\equiv {\overline{d}}_n\left({\widehat{\mathsf{\theta }}}_n\right)$. Let ${\overline{d}}_n^{\ast }\equiv {\overline{d}}_n\left({\mathsf{\theta }}^{\ast }\right)$.

Let $d\left(\mathsf{\theta }\right)\equiv \left[\begin{array}{c}vech\left(A\left(\mathsf{\theta }\right)\right)\\ vech\left(B\left(\mathsf{\theta }\right)\right)\end{array}\right]$. Let $d^{\ast }\equiv d\left({\mathsf{\theta }}^{\ast }\right)$.

Let the notation $I_k$ denote a k-dimensional identity matrix.

Let the duplication matrix ${\mathcal{D}}_k:{\mathcal{R}}^{k(k+1)/2}\to {\mathcal{R}}^{k^2}$ be defined such that: ${\mathcal{D}}_{k}vech(A)=vec(A)$ and the inverse duplication matrix ${\mathcal{D}}_k^{†}:{\mathcal{R}}^{k^2}\to {\mathcal{R}}^{k(k+1)/2}$ be defined such that: ${\mathcal{D}}_k^{†}vec(A)=vech(A)$.

Let ${\mathcal{D}}_k^{\otimes }\equiv I_2\otimes {\mathcal{D}}_k$ and let ${\mathcal{D}}_k^{\otimes †}\equiv I_2\otimes {\mathcal{D}}_k^{†}$.

Let $\mathsf{\nabla }d:\mathsf{\Theta }\to {\mathcal{R}}^{k(k+1)}$ where $\mathsf{\nabla }d\equiv \left[\begin{array}{c}\frac{dvech\left(A\right)}{d\mathsf{\theta }}\\ \frac{dvech\left(B\right)}{d\mathsf{\theta }}\end{array}\right]={\mathcal{D}}_k^{\otimes †}\left[\begin{array}{c}\frac{dA}{d\mathsf{\theta }}\\ \frac{dB}{d\mathsf{\theta }}\end{array}\right]$.

Let $\mathsf{\nabla }{\overline{d}}_n\left(\mathsf{\theta }\right)\equiv (1/n){\displaystyle \sum _{i=1}^n\mathsf{\nabla }{d}_{x,\mathsf{\theta }}}\left(X_i;\mathsf{\theta }\right)$. Let $\mathsf{\nabla }{\widehat{d}}_n\equiv \mathsf{\nabla }{\overline{d}}_n\left({\widehat{\mathsf{\theta }}}_n\right)$. Let $\mathsf{\nabla }{d}^{\ast }\equiv \mathsf{\nabla }d\left({\mathsf{\theta }}^{\ast }\right)$.

2.5. Regularity Conditions

The following Assumption 5 uses a matrix version of the standard definition of dominated by an integrable function (see Appendix A).

Assumption 5.

Domination Conditions

(i)(a) $\log f\left(x;\mathsf{\theta }\right)$ is dominated on Θ with respect to $p_x$;

(i)(b) $\frac{d\log f\left(x;\mathsf{\theta }\right)}{d\mathsf{\theta }}$ is dominated on Θ with respect to $p_x$;

(i)(c) $g\left(x;\mathsf{\theta }\right){\left(g\left(x;\mathsf{\theta }\right)\right)}^T$ is dominated on Θ with respect to $p_x$;

(i)(d) $\frac{d^2\log f\left(x;\mathsf{\theta }\right)}{d{\mathsf{\theta }}^2}$ is dominated on Θ with respect to $p_x$;

(ii)(a) $\frac{d\left(d_{x,\mathsf{\theta }}\left(x;\mathsf{\theta }\right)\right)}{d\mathsf{\theta }}$ is dominated on Θ with respect to $p_x$;

(ii)(b) $d_{x,\mathsf{\theta }}\left(x;\mathsf{\theta }\right){\left(d_{x,\mathsf{\theta }}\left(x;\mathsf{\theta }\right)\right)}^T$ is dominated on Θ with respect to $p_x$;

(ii)(c) $g\left(x;\mathsf{\theta }\right){\left(d_{x,\mathsf{\theta }}\left(x;\mathsf{\theta }\right)\right)}^T$ is dominated on Θ with respect to $p_x$;

(iii) There exists a finite positive number K such that for all $x\in suppX$ and for all $\mathsf{\theta }\in \mathsf{\Theta }$ : $\left|\frac{f\left(x;\mathsf{\theta }\right)}{p_x\left(x\right)}\right|\le K$.

Assumption 5 identifies specific regularity conditions that are used here to ensure that relevant expectations exist, that integral and differentiation operators can be interchanged, and that relevant laws of large numbers are applicable.

Assumption 5(i) is used to ensure that the conclusions of Theorems 2, 3, 4, 5, 6, and 7 hold. These theorems characterize the asymptotic distribution of the quasi-maximum likelihood estimator. Assumption 5(ii) is also required to ensure that the conclusions of Theorems 6 and 7 hold which characterize the asymptotic distribution of $s\left({\widehat{A}}_n,{\widehat{B}}_n\right)$.

A sufficient but not necessary condition for both Assumption 5(i) and Assumption 5(ii) to hold is that $\log f$ is thrice continuously differentiable on the compact set Θ, measurable in its first argument (e.g., piecewise continuous), and the support of X is bounded. The assumption that the support of X is bounded is satisfied, for example, by observational data consisting of discrete random variables. Assumptions 5(i) and Assumption 5(ii) more generally are satisfied for many commonly used finite-dimensional parametric smooth probability models for observational data modeled as combinations of both discrete and absolutely continuous random variables.

Assumption 5(iii) in conjunction with Assumptions 5(i) and Assumption 5(ii) is used in Theorem 4 to ensure that: (1) when $A^{\ast }\ne B^{\ast }$ this corresponds to the case of model misspecification; and (2) the correctly specified probability model implies that $A^{\ast }=B^{\ast }$. Thus, Assumption 5(iii) is important for ensuring the proper semantic interpretation of a GIMT result (see Proposition 1 and Theorem 4). In addition, Assumption 5(iii) in conjunction with Assumption 5(i) and Assumption 5(ii) is also used to ensure that the Lancaster-Chesher approximation holds (see Theorem 8), which provides a method for constructing GIMTs without computing the third derivative of the negative log-likelihood function.

Assumption 5(iii) can be interpreted as stating that the density $f\left(x;\mathsf{\theta }\right)$ in the probability model and the data generating process density $p_x\left(x\right)$ cannot be too dissimilar. A sufficient but not necessary condition for satisfying Assumption 5(iii) would be that there exists two finite positive numbers $K_1$ and $K_2$ such that for all $\mathsf{\theta }\in \mathsf{\Theta }$ and for all $\mathit{x}\in \mathrm{supp}\mathit{X}$: $f\left(x;\mathsf{\theta }\right)<K_1$ and $p_x\left(x\right)>K_2$. Although Assumption 5(iii) could be formulated in a slightly more general manner, we use this more specialized version for expository reasons.

The negative average log-likelihood is defined as:

$${\overline{l}}_n\left(\mathsf{\theta }\right)\equiv -n^{-1}{\displaystyle \sum _{i=1}^n\log f\left(X_i;\mathsf{\theta }\right)}.$$

When it exists, the unique global minimizer of ${\overline{l}}_n\left(\mathsf{\theta }\right)$ is called the quasi-maximum likelihood estimate ${\widehat{\mathsf{\theta }}}_n$ rather than a maximum likelihood estimate to allow for the possibility that f may be misspecified [1].

The negative expected log-likelihood is defined as:

$$l\left(\mathsf{\theta }\right)\equiv -{\displaystyle \int p_x\left(x\right)\log f\left(x;\mathsf{\theta }\right)d\nu \left(x\right)}.$$

A global minimizer of $l\left(\mathsf{\theta }\right)$ is called the pseudo-true parameter value ${\mathsf{\theta }}^{\ast }$ because of the possibility that f may be misspecified. If there exists a ${\mathsf{\theta }}_o$ such that $f\left(\cdot ;{\mathsf{\theta }}_0\right)=p_x$ $\nu \left(x\right)$ almost everywhere, then ${\mathsf{\theta }}_o$ is called a true parameter value.

Assumption 6.

Uniqueness. (i) For some ${\mathsf{\theta }}^{\ast }\in \mathsf{\Theta }$, $l$ has a unique minimum at ${\mathsf{\theta }}^{\ast }$; (ii) ${\mathsf{\theta }}^{\ast }$ is interior to Θ.

Let $H_0:s\left(A^{\ast },B^{\ast }\right)=0_r$ be a particular GIMT null hypothesis specified by a given GIMT hypothesis function s. Our ultimate goal is to construct a statistical test for testing the GIMT null hypothesis $H_0:s\left(A^{\ast },B^{\ast }\right)=0_r$ by characterizing the asymptotic behavior of the test statistic ${\widehat{s}}_n\equiv s\left({\widehat{A}}_n,{\widehat{B}}_n\right)$. Note that the GIMT hypothesis function test statistic ${\widehat{s}}_n\equiv s\left({\widehat{A}}_n,{\widehat{B}}_n\right)$ is an estimator of $s^{\ast }\equiv s\left(A^{\ast },B^{\ast }\right)$ (see Theorem 6).

Let $\mathsf{\nabla }{s}^{\ast }\equiv \mathsf{\nabla }s\left(A^{\ast },B^{\ast }\right)$. Let $\mathsf{\nabla }{\widehat{s}}_n\equiv \mathsf{\nabla }s\left({\widehat{A}}_n,{\widehat{B}}_n\right)$.

Let ${\mathsf{\delta }}^{\ast }\left(X_i\right)\equiv \mathsf{\nabla }{s}^{\ast }{\mathcal{D}}_k^{\otimes }\left(d_{x,\mathsf{\theta }}\left(X_i;{\mathsf{\theta }}^{\ast }\right)-\mathsf{\nabla }{d}^{\ast }{\left(A^{\ast }\right)}^{-1}g\left(X_i;{\mathsf{\theta }}^{\ast }\right)-d^{\ast }\right)$.

Given appropriate regularity conditions, it will be shown (see Theorems 6 and 7) that the asymptotic covariance matrix of $n^{1/2}\left({\widehat{s}}_n-s^{\ast }\right)$ is the GIMT asymptotic covariance matrix

$${\Sigma }_s^{\ast }\equiv {\displaystyle \mathbf{\int }{\mathsf{\delta }}^{\ast }\left(x_i\right){\left({\mathsf{\delta }}^{\ast }\left(x_i\right)\right)}^T{p}_x\left(x_i\right)d\nu \left(x_i\right)},$$

(1)

which may be estimated by

$${\widehat{\Sigma }}_s^n\equiv (1/n){\displaystyle \sum _{i=1}^n{\widehat{\mathsf{\delta }}}_n\left(X_i\right){\left({\widehat{\mathsf{\delta }}}_n\left(X_i\right)\right)}^T},$$

(2)

where ${\widehat{\mathsf{\delta }}}_n\left(X_i\right)\equiv \mathsf{\nabla }{\widehat{s}}_n{\mathcal{D}}_k^{\otimes }\left(d_{x,\mathsf{\theta }}\left(X_i;{\widehat{\mathsf{\theta }}}_n\right)-\mathsf{\nabla }{\widehat{d}}_n{\left({\widehat{A}}_n\right)}^{-1}g\left(X_i;{\widehat{\mathsf{\theta }}}_n\right)-{\widehat{d}}_n\right)$.

Assumption 7.

Positive Definiteness. (i) $A^{\ast }$ is positive definite; (ii) $B^{\ast }$ is positive definite; and (iii) ${\sum }_s^{\ast }$ is positive definite.

Assumption 7(i) is a sufficient but not necessary condition for the quasi-maximum likelihood estimate to be a strict local minimizer. Assumption 7(ii) is used in order to apply the Multivariate Central Limit Theorem to characterize the asymptotic distribution of the quasi-maximum likelihood estimates. Assumption 7(iii) is used in order to apply the Multivariate Central Limit Theorem to obtain the asymptotic distribution of the GIMT statistic ${\widehat{s}}_n$. Violation of Assumption 7 is analogous to the presence of multicollinearity in classical linear regression modeling.

Assumptions 6, 7(i), and 7(ii) are often checked in practice by checking if the infinity norm of ${\widehat{g}}_n$ is sufficiently small and that the condition numbers of ${\widehat{A}}_n$ and ${\widehat{B}}_n$ are not excessively large. In addition, it is necessary to check that the condition number of an estimator of ${\sum }_s^{\ast }$ denoted by ${{\displaystyle \widehat{\sum }}}_s^n$ (see Equation (2)) is not excessively large. Note that Assumption 4(iv) is a necessary condition for ${\sum }_s^{\ast }$ to be positive definite. If the magnitude of the asymptotic covariance matrix of the selection test statistic ${\widehat{s}}_n\equiv s\left({\widehat{A}}_n,{\widehat{B}}_n\right)$, ${\sum }_s^{\ast }$, is not finite or ${\sum }_s^{\ast }$ is singular, then Assumption 7(iii) fails.

3. GIMT Theoretical Framework: Theorems and Formulas

In this section, a brief review of relevant results from classical asymptotic theory is provided (Theorems 1, 2, 3, 4, 5, 8) in conjunction with our new results in Theorems 6 and 7. Proofs of all theorems and propositions are provided in the Appendix A.

3.1. Classical Results

Theorem 1.

Estimator Measurability ([30], Lemma 2). Assume that Assumptions 1, 2, 3(i), and 3(ii) hold. Let $P_0^n$ be the joint distribution of $X_1,\mathrm{...},X_n$. Then for each n = 1, 2, … , there exists a measurable function ${\widehat{\mathsf{\theta }}}_n:{\mathcal{R}}^{dn}\to \mathsf{\Theta }$ and an element, B_n, of ${\left(\mathcal{B}\left({\mathcal{R}}^d\right)\right)}^n$ with $P_0^n\left(B_n\right)=1$ such that for all $\left\{x_1,\dots ,x_n\right\}\in B_n$:

$${\overline{l}}_n\left({\widehat{\mathsf{\theta }}}_n\left(\left\{x_1,\dots ,x_n\right\}\right)\right)=\underset{\mathsf{\theta }\in \mathsf{\Theta }}{\min }{\overline{l}}_n\left(\mathsf{\theta }\right).$$

Theorem 2.

Estimator Consistency ([31], Theorem 2.1). Assume Assumptions 1, 2, 3(i), 3(ii), 5(i)a, and 6 hold. Then as $n\to \infty$, ${\widehat{\mathsf{\theta }}}_n\to {\mathsf{\theta }}^{\ast }$ with probability one.

Theorem 3.

Estimator Asymptotic Distribution ([1], Theorem 3.2; also see [32]). Assume Assumptions 1, 2, 3(i), 3(ii), 3(iii), 3(iv), 5(i), 6, 7(i), and 7(ii) hold. As n → ∞, $\sqrt{n}\left({\widehat{\mathsf{\theta }}}_n-{\mathsf{\theta }}^{\ast }\right)$ converges in distribution to a zero-mean Gaussian random vector with non-singular covariance matrix $C^{\ast }\equiv {\left(A^{\ast }\right)}^{-1}{B}^{\ast }{\left(A^{\ast }\right)}^{-1}$.

Theorem 4.

Contrapositive Information Matrix Equality ([1], Theorem 3.3). Assume Assumptions 1, 2, 3(i), 3(ii), 3(iii), 3(iv), 5, and 6 hold. If $A^{\ast }\ne B^{\ast }$, then the probability model $\mathcal{M}\equiv \left\{F\left(\cdot ;\mathsf{\theta }\right):{\mathcal{R}}^d\to [0,1]|\mathsf{\theta }\in \mathsf{\Theta }\right\}$ is misspecified.

Theorem 4 is the contrapositive statement of the familiar information matrix equality that states that if a smooth regular probability model is correctly specified, then $A^{\ast }=B^{\ast }$. The contrapositive statement implies that a difference between $A^{\ast }$ and $B^{\ast }$ indicates the presence of model misspecification.

Moreover, if the information matrix equality is violated (i.e., $A^{\ast }\ne B^{\ast }$), then the asymptotic distribution of the quasi-maximum likelihood estimator is still Gaussian centered at ${\mathsf{\theta }}^{\ast }$ but its asymptotic covariance matrix is $C^{\ast }\equiv {\left(A^{\ast }\right)}^{-1}{B}^{\ast }{\left(A^{\ast }\right)}^{-1}$. In this case, the standard formulas for estimating the asymptotic covariance matrix of the maximum likelihood estimators based upon estimating either ${\left(A^{\ast }\right)}^{-1}$ or ${\left(B^{\ast }\right)}^{-1}$ are not appropriate. Thus, detecting that $A^{\ast }\ne B^{\ast }$ is not only useful for detecting model misspecification but also detects situations where the sandwich covariance matrix estimator ${\widehat{C}}_n\equiv {\left({\widehat{A}}_n\right)}^{-1}{\widehat{B}}_n{\left({\widehat{A}}_n\right)}^{-1}$ should be used to ensure an asymptotically unbiased estimate of $C^{\ast }\equiv {\left(A^{\ast }\right)}^{-1}{B}^{\ast }{\left(A^{\ast }\right)}^{-1}$ is obtained. This is important in applications when one encounters predictive, yet misspecified, models. For example, a linear regression model may have small residual errors yet the residual error term is not Gaussian.

Let ${\overline{C}}_n\left(\mathsf{\theta }\right)\equiv {\left({\overline{A}}_n\right)}^{-1}{\overline{B}}_n\left(\mathsf{\theta }\right){\left({\overline{A}}_n\left(\mathsf{\theta }\right)\right)}^{-1}$. Let ${\widehat{C}}_n\equiv {\overline{C}}_n\left({\widehat{\mathsf{\theta }}}_n\right)$.

Theorem 5.

Consistent QMLE Covariance Matrix Estimators (e.g., [1]). Assume Assumptions 1, 2, 3(i), 3(ii), 3(iii), 3(iv), 5(i), 6, 7(i) and 7(ii) hold. Then, with probability one as $n\to \infty$: ${\widehat{B}}_n\to B^{\ast }$, ${\left({\widehat{B}}_n\right)}^{-1}\to {\left(B^{\ast }\right)}^{-1}$, ${\widehat{A}}_n\to A^{\ast }$, ${\left({\widehat{A}}_n\right)}^{-1}\to {\left(A^{\ast }\right)}^{-1}$, ${\widehat{C}}_n\to C^{\ast }$, and ${\left({\widehat{C}}_n\right)}^{-1}\to {\left(C^{\ast }\right)}^{-1}$.

3.2. GIMT Statistic Asymptotic Behavior

Theorem 6.

GIMT Statistic Consistency. Assume Assumptions 1, 2, 3, 4(i), 4(ii), 4(iii), 5(i), and 6 hold. Then as $n\to \infty$, ${\widehat{s}}_n\to s^{\ast }$ with probability one. If, in addition, Assumptions 5(ii) and 7(iii) hold, then with probability one ${{\displaystyle \widehat{\sum }}}_s^n\to {\sum }_s^{\ast }$ and ${\left({{\displaystyle \widehat{\sum }}}_s^n\right)}^{-1}\to {\left({\sum }_s^{\ast }\right)}^{-1}$ as $n\to \infty$.

The asymptotic distribution of ${\widehat{s}}_n\equiv s\left({\widehat{A}}_n,{\widehat{B}}_n\right)$ is described in the next theorem. Strategies for estimating ${\sum }_s^{\ast }$ are discussed at the end of this section.

Theorem 7.

Generalized Information Matrix Wald Test. Assume Assumptions 1, 2, 3, 4, 5(i), 5(ii), 6, 7 hold with respect to a GIMT hypothesis function $s:\mathsf{\Upsilon }\times \mathsf{\Upsilon }\to {\mathcal{R}}^r$ and probability model $\mathcal{M}$. Let ${\widehat{\mathcal{W}}}_n\equiv n{\left({\widehat{s}}_n\right)}^T{\left({{\displaystyle \widehat{\sum }}}_s^n\right)}^{-1}\left({\widehat{s}}_n\right)$. If $H_0:s^{\ast }=0_r$, then ${\widehat{\mathcal{W}}}_n\stackrel{d}{\to }{\chi }_r^2$ as $n\to \infty$. If $H_0:s^{\ast }=0_r$ is false, then ${\widehat{\mathcal{W}}}_n\to \infty$ as $n\to \infty$ w.p.1.

Using a Wald test approach, Theorem 7 establishes that the GIMT p-value will be consistently estimated under the null hypothesis $H_0:s^{\ast }=0_r$ thus allowing us to bound Type 1 errors by chosen significance levels. Under the alternative hypothesis $H_a:s^{\ast }\ne 0_r$, Theorem 7 ensures that the Type 2 error goes to zero as the sample size increases with probability one.

From Theorem 4 and the definition of a GIMT Hypothesis Function $s:\mathsf{\Upsilon }\times \mathsf{\Upsilon }\to {\mathcal{R}}^r$, it follows that $s\left(A^{\ast },B^{\ast }\right)\ne 0_r$ implies the presence of model misspecification. This statement follows immediately from the definition of a GIMT hypothesis function and the conclusion of Theorem 4. It is formally presented because of its semantic importance.

Proposition 1.

Interpretation of GIMT Null and Alternative Hypotheses. Suppose the Assumptions of Theorem 4 hold. Let s be a GIMT hypothesis function. (i) If $\mathcal{M}$ is correctly specified, then $H_0:s^{\ast }=0_r$; (ii) If $H_0:s^{\ast }=0_r$ is false, then $\mathcal{M}$ is misspecified.

Proposition 1 states that for either a directional or nondirectional GIMT, evidence supporting the rejection of the null hypothesis $H_0:s^{\ast }=0_r$ is also evidence supporting the presence of model misspecification. Note, however, the assertion that $H_0:s^{\ast }=0_r$ is true does not necessarily imply correct model specification.

3.3. GIMT Covariance Matrix Estimators

A non-directional GIMT covariance matrix estimator ${\ddot{\Sigma }}_s^n$ is defined as an estimator with the following two properties: (i) ${\ddot{\Sigma }}_s^n\to {\Sigma }_s^{\ast }$ as $n\to \infty$ with probability one when $A^{\ast }=B^{\ast }$; and (ii) ${\ddot{\Sigma }}_s^n$ converges to a positive definite matrix as $n\to \infty$ with probability one regardless of whether the probability model is correctly specified. Property (ii) is analogous to Assumption 7(iii) and can be empirically checked by examining the condition number of the GIMT covariance matrix estimator ${\ddot{\Sigma }}_s^n$.

Let the Lancaster-Chesher 3rd Derivative Formula $\ddot{\mathsf{\nabla }}d:\mathsf{\Theta }\to {\mathcal{R}}^{k(k+1)\times k}$ be defined such that:

$$\ddot{\mathsf{\nabla }}{d}_n\left(\mathsf{\theta }\right)={\mathcal{D}}_k^{\otimes †}\left[\begin{array}{c}\frac{d{\overline{B}}_n\left(\mathsf{\theta }\right)}{d\mathsf{\theta }}+n^{-1}{\displaystyle \sum _{i=1}^{n}vec\left(A\left(X_i;\mathsf{\theta }\right)-B\left(X_i;\mathsf{\theta }\right)\right){\left(g\left(X_i;\mathsf{\theta }\right)\right)}^T}\\ \frac{d{\overline{B}}_n\left(\mathsf{\theta }\right)}{d\mathsf{\theta }}\end{array}\right],$$

(3)

where

$$\frac{d{\overline{B}}_n\left(\mathsf{\theta }\right)}{d\mathsf{\theta }}=n^{-1}{\displaystyle \sum _{i=1}^n\left[\left(A\left(X_i;\mathsf{\theta }\right)\otimes g\left(X_i;\mathsf{\theta }\right)\right)+\left(g\left(X_i;\mathsf{\theta }\right)\otimes A\left(X_i;\mathsf{\theta }\right)\right)\right]}.$$

(4)

The formulas for the GIMT covariance matrix estimator require computation of both the second and third derivatives of the negative log-likelihood function, which are represented in Equations (1) and (2) by the formula $\mathsf{\nabla }{d}^{\ast }$. Theorem 8 shows that the formula $\ddot{\mathsf{\nabla }}{\widehat{d}}_n\equiv \ddot{\mathsf{\nabla }}{d}_n\left({\widehat{\mathsf{\theta }}}_n\right)$, which uses only first and second derivatives of the negative average log-likelihood, may be used to asymptotically approximate $\mathsf{\nabla }{d}^{\ast }$ for the purpose of avoiding calculation of negative average log-likelihood third derivatives.

Theorem 8.

Lancaster-Chesher Estimator (see [12]). Assume Assumptions 1, 2, 3, 5(i)a, 5(i)c, 5(i)d, 5(ii)a, 5(ii)c, 5(iii), and 6 hold with respect to a GIMT hypothesis function $s:\mathsf{\Upsilon }\times \mathsf{\Upsilon }\to {\mathcal{R}}^r$ and probability model $\mathcal{M}$. If $\mathcal{M}$ is correctly specified, then with probability one $\ddot{\mathsf{\nabla }}{d}_n\left({\widehat{\mathsf{\theta }}}_n\right)\to \mathsf{\nabla }{d}^{\ast }$ as $n\to \infty$.

Theorem 8 provides an additional mechanism for constructing alternative and possibly computationally convenient covariance matrix estimators for estimating ${\Sigma }_s^{\ast }$ when the null hypothesis that the model is correctly specified holds. In particular, the formula $\ddot{\mathsf{\nabla }}{d}_n\left({\widehat{\mathsf{\theta }}}_n\right)$ is substituted for $\mathsf{\nabla }{\widehat{d}}_n$ in Equation (2) to obtain a real symmetric matrix with non-negative eigenvalues called the Lancaster-Chesher covariance matrix estimator. If the null hypothesis that the model is correctly specified is false, then the Lancaster-Chesher covariance matrix estimator simply needs to converge to any finite positive definite matrix. This latter assumption can be empirically checked by examining the condition number of the Lancaster-Chesher covariance matrix estimator.

We now provide formulas for a variety of different types of non-directional GIMT covariance matrix estimators. First note that when the probability model is correctly specified, the contrapositive of Theorem 4 in conjunction with Theorem 5 implies that ${\widehat{A}}^{-1}={\widehat{B}}^{-1}=\widehat{C}$. Thus, one can use either the OPG inverse Hessian estimator ${\widehat{B}}^{-1}$ or the sandwich inverse Hessian estimator ${\widehat{C}}_n$ as alternative estimators for the inverse Hessian estimator ${\widehat{A}}^{-1}$ in (2). Second, if the GIMT selection function s is anti-symmetric and $A^{\ast }=B^{\ast }$, then it follows that the term $\left(\mathsf{\nabla }{s}^{\ast }\right){\mathcal{D}}_k^{\otimes }{d}^{\ast }=0_r$ so that the centering term $d^{\ast }$ in (1) can be set equal to $0_r$. Thus, an alternative estimator of $d^{\ast }$ that can be used instead of the centering term estimator ${\widehat{d}}_n$ in (2) is simply a vector of zeros. These two methods yield six different non-directional GIMT covariance matrix estimators.

Six additional GIMT covariance matrix estimators can be obtained by using the Lancaster-Chesher estimator $\ddot{\mathsf{\nabla }}{\widehat{d}}_n$ (defined above) as an alternative estimator for the third derivative negative average log-likelihood estimator $\mathsf{\nabla }{\widehat{d}}_n$. The Lancaster-Chesher estimator $\ddot{\mathsf{\nabla }}{\widehat{d}}_n$ has the computational advantage relative to $\mathsf{\nabla }{\widehat{d}}_n$ that only the first and second derivatives of the negative log-likelihood are used. However, previous empirical studies have suggested that the use of the Lancaster-Chesher estimator $\ddot{\mathsf{\nabla }}{\widehat{d}}_n$ instead of the third-derivative negative average log-likelihood estimator $\mathsf{\nabla }{\widehat{d}}_n$ may degrade performance in some cases (e.g., [13,15,16,17,18]).

3.4. Adjusted GIMT Hypothesis Functions

Assumption 7(iii) requires that ${\Sigma }_s^{\ast }$ is a positive definite matrix. The GIMT selection function s may have the property that the r-dimensional matrix ${\Sigma }_s^{\ast }$ is singular with rank g where g < r so that Assumption 7(iii) fails. However, it is often possible to replace the original GIMT hypothesis function $s:\mathsf{\Upsilon }\times \mathsf{\Upsilon }\to {\mathcal{R}}^r$ with an alternative “adjusted” GIMT hypothesis function $s^{\prime }:\mathsf{\Upsilon }\times \mathsf{\Upsilon }\to {\mathcal{R}}^g$ that tests a similar null hypothesis yet has the properties that: (i) the resulting asymptotic covariance matrix of $n^{1/2}{\mathbf{s}}_n^{\prime }$ is nonsingular; and (ii) rejection of $H_0:s^{\prime }(A^{\ast },B^{\ast })=0_r$ implies rejection of $H_0:s(A^{\ast },B^{\ast })=0_r$.

Proposition 2.

Adjusted GIMT Hypothesis Function Properties. Let ${\Sigma }_s^{\ast }$ be an r-dimensional GIMT asymptotic covariance matrix for GIMT hypothesis function $s:\mathsf{\Upsilon }\times \mathsf{\Upsilon }\to {\mathcal{R}}^r$ such that Assumption 7(iii) holds. Let the $g$ rows of the rank g matrix $T\in {\mathcal{R}}^{g\times r}$ be r-dimensional orthonormal eigenvectors of ${\Sigma }_s^{\ast }$ $\left(r>g\ge 1\right)$ for GIMT hypothesis function s. Define an alternative GIMT hypothesis function $s^{\prime }\equiv Ts$ whose respective g-dimensional GIMT asymptotic covariance matrix is ${\Sigma }_T^{\ast }=T{\Sigma }_s^{\ast }{T}^T$. (i) If $H_0:s^{\prime }\left(A^{\ast },B^{\ast }\right)=0_g$ is false, then $H_0:s\left(A^{\ast },B^{\ast }\right)=0_r$ is false; (ii) The g-dimensional GIMT asymptotic covariance matrix, ${\Sigma }_T^{\ast }$, for $s^{\prime }$ is finite and positive definite.

The matrix T in Proposition 1 is called the adjusted GIMT hypothesis projection matrix. The proof of Proposition 2(i) follows from the observation that if ${\left|s\right|}^2=0_r$, then ${\left|s\prime \right|}^2=0_g$. Proposition 2(ii) follows from the observation that ${\Sigma }_T^{\ast }=T{\Sigma }_s^{\ast }{T}^T$ is non-singular by the construction of T and Assumption 7(iii).

4. Simulation Studies

As discussed, some previously published information matrix tests for model misspecification have demonstrated good level and power performance (e.g., [19,23,24]). These tests may be viewed with respect to the GIMT framework presented here. The theoretical framework presented in Section 2 and Section 3 provides an important perspective in understanding the similarities and differences among existing misspecification tests within a unified framework. Further, these prior published empirical studies support the value of the GIMT framework by showing that GIMTs with good level and power performance can be constructed.

However, the GIMT framework in Section 2 and Section 3 is also valuable for developing entirely new GIMTs for a large class of probability models in a straightforward manner through the use of Theorems 6 and 7. To illustrate our approach to the construction and evaluation of such GIMTs, we show how Theorems 6 and 7 can be used to derive five new GIMTs. Although an important goal of these derivations was an interest in developing useful tests for model misspecification, a major reason for deriving five additional GIMTs was to demonstrate the flexibility and generality of the unified GIMT theory developed in Section 2 and Section 3 .

Next, simulation studies of the level and power performance of the new GIMTs are provided to examine the performance of the GIMTs for some specific empirical examples. This particular logistic regression modeling problem is intended to be representative of a commonly encountered situation where a relevant predictor in a regression model is not properly recoded and an irrelevant predictor is included. The simulation studies were not intended to be comprehensive but rather were designed to empirically demonstrate how the general GIMT theory (Section 2 and Section 3) can be used to develop a wide range of misspecification tests. For comparison purposes, the Adjusted Classical GIMT originally proposed by Golden et al. [23] was included as a sixth GIMT in the simulation studies.

4.1. Generalized Information Matrix Tests

4.1.1. Adjusted Classical GIMT (Directional) [23]

Suppose one desires to test the classical full Information Matrix Test hypothesis $H_0:A^{\ast }=B^{\ast }$. Let ${\Sigma }_s^{\ast }$ be the r-dimensional GIMT asymptotic covariance matrix associated with this GIMT. Note that $r=k\left(k+1\right)/2$ may be relatively large. Assume, however, that ${\Sigma }_s^{\ast }$ only has rank g where g < r. Because ${\Sigma }_s^{\ast }$ is not of full rank, the asymptotic theory developed here cannot be directly applied since Assumption 7(iii) is violated. However, following the discussion in Proposition 1, let $T\in {\mathcal{R}}^{g\times r}$ be a matrix with full row rank defined such that the g rows of T are r-dimensional orthonormal eigenvectors of ${\Sigma }_s^{\ast }$ ($r>g\ge 1$). Then, instead of testing the null hypothesis $H_0:A^{\ast }=B^{\ast }$ associated with the classical full non-directional Information Matrix Test [1], the null hypothesis $H_0:Tvech\left(A^{\ast }\right)=Tvech\left(B^{\ast }\right)$ is tested using the GIMT hypothesis function s defined such that: $s\left(A^{\ast },B^{\ast }\right)=Tvech\left(A^{\ast }-B^{\ast }\right)$. The GIMT associated with this hypothesis function is called the Adjusted Classical GIMT (Directional). Golden et al. [23] provided further discussion of this GIMT and showed that it had good level and power properties using simulation studies of a realistic epidemiological data analysis problem.

4.1.2. Fisher Spectra GIMT (Directional)

The Fisher Spectra GIMT (Directional) is a new k-degree of freedom test specified by the GIMT hypothesis function s defined such that:

$$s\left(A^{\ast },B^{\ast }\right)=diag\left({\left(A^{\ast }\right)}^{-1}{B}^{\ast }\right)-1_k,$$

which tests the null hypothesis $H_o:s\left(A^{\ast },B^{\ast }\right)=0_k.$ The notation $1_k$ denotes a k-dimensional column vector of ones. The $diag:{\mathcal{R}}^{k\times k}\to {\mathcal{R}}^k$ is defined such that $diag\left({\left(A^{\ast }\right)}^{-1}{B}^{\ast }\right)$ is a column vector of the on-diagonal elements of ${\left(A^{\ast }\right)}^{-1}{B}^{\ast }$. The degrees of freedom of this test are equal to the number of free parameters in the model. When the Information Matrix Equality holds, then ${\left(A^{\ast }\right)}^{-1}{B}^v$ will be the identity matrix and this GIMT tests the null hypothesis that the k on-diagonal elements of ${\left(A^{\ast }\right)}^{-1}{B}^{\ast }$ are all equal to one. Note that the Fisher Spectra GIMT tests that the eigenvalues of the two matrices are the same but does not test the null hypothesis that the two matrices have the same eigenvectors. The Fisher Spectra GIMT presented here is similar to the Copula Eigenvalue Test [33]; however, the test statistic is different because the Fisher Spectra GIMT was not developed within a copula framework.

4.1.3. Robust Log GAIC GIMT (Directional)

The Robust Log GAIC GIMT (Directional) is a new 1-degree of freedom test specified by the GIMT hypothesis function s defined such that:

$$s\left(A^{\ast },B^{\ast }\right)=\log \left(\left(\frac{1}{k}\right)trace\left({\left(A^{\ast }\right)}^{-1}{B}^{\ast }\right)\right),$$

which tests the null hypothesis $H_o:s\left(A^{\ast },B^{\ast }\right)=0.$ If the null hypothesis of this test is rejected, then not only does it indicate the presence of model misspecification, it mandates that one uses misspecification-robust estimation methods such as the sandwich estimator [1,32] and misspecification-robust model selection criteria such as the Generalized Akaike Information Criterion (GAIC) [34,35,36]. The GAIC that is defined by the formula: $GAIC=2n{\widehat{l}}_n+2trace\left({\left(A^{\ast }\right)}^{-1}{B}^{\ast }\right)$ is an unbiased estimator of the expected value of the log-likelihood measure $2n{\widehat{l}}_n$ (e.g., see Appendix of [35]). Note that the Log GAIC GIMT tests the same null hypothesis as the IOS IMT described by Presnell and Boos [19] (also see [20,21,22]); however, the test statistic is the logarithm of the IOS IMT statistic.

4.1.4. Robust Log GAIC Ratio GIMT (Directional)

The 1-degree of freedom Composite Log GAIC Ratio GIMT is specified by the GIMT hypothesis function s is defined such that:

$$s\left(A^{\ast },B^{\ast }\right)=\log \left(\frac{trace\left({\left(A^{\ast }\right)}^{-1}{B}^{\ast }\right)}{trace\left({\left(B^{\ast }\right)}^{-1}{A}^{\ast }\right)}\right),$$

which tests the null hypothesis $H_o:s\left(A^{\ast },B^{\ast }\right)=0.$ The Robust Log GAIC Ratio GIMT (Directional) tests a null hypothesis similar to the null hypotheses associated with the group of non-directional 1 degree of freedom GIMTs discussed by Cho and Phillips [37] that compares the arithmetic mean and harmonic mean of the eigenvalues of the matrix ${\left(A^{\ast }\right)}^{-1}{B}^{\ast }$. It is also closely related to the IOS IMT discussed by Presnell and Boos [19] (also see [20,21,22]).

4.1.5. Composite Log GAIC GIMT (Nondirectional)

Lemma 1 of [37] shows that $A^{\ast }=B^{\ast }$ if and only if $trace\left({(A^{\ast })}^{-1}{B}^{\ast }\right)=k\mathrm{and}trace\left({(B^{\ast })}^{-1}{A}^{\ast }\right)=k$. This result provides a justification for a new type of GIMT called the 2-degree of freedom Composite Log GAIC GIMT (Non-Directional). The Composite Log GAIC GIMT specified by the GIMT hypothesis function s is defined such that:

$$s\left(A^{\ast },B^{\ast }\right)=\left[\begin{array}{c}\log \left(\left(\frac{1}{k}\right)trace\left({\left(A^{\ast }\right)}^{-1}{B}^{\ast }\right)\right)\\ \log \left(\left(\frac{1}{k}\right)trace\left({\left(B^{\ast }\right)}^{-1}{A}^{\ast }\right)\right)\end{array}\right],$$

which tests the null hypothesis $H_o:s\left(A^{\ast },B^{*}\right)=0_2.$ The Composite Log GAIC GIMT (Non-Directional) tests a null hypothesis similar to the null hypotheses associated with the group of non-directional 1 degree of freedom GIMTs discussed by Cho and Phillips [37] that compare the arithmetic mean and harmonic mean of the eigenvalues of the matrix ${\left(A^{\ast }\right)}^{-1}{B}^{\ast }$.

4.1.6. Composite GAIC GIMT (Non-Directional)

The Composite GAIC GIMT (Non-Directional) tests exactly the same null hypothesis as the Composite Log GAIC GIMT but does not include the log transformation. The Composite GAIC GIMT specified by the GIMT hypothesis function s is defined such that:

$$s\left(A^{\ast },B^{\ast }\right)=\left[\begin{array}{c}\left(\frac{1}{k}\right)trace\left({\left(A^{\ast }\right)}^{-1}{B}^{\ast }\right)-1\\ \left(\frac{1}{k}\right)trace\left({\left(B^{\ast }\right)}^{-1}{A}^{\ast }\right)-1\end{array}\right],$$

which tests the null hypothesis $H_o:s\left(A^{\ast },B^{\ast }\right)=0_2.$ Cho and Phillips [37] have proposed the magnitude of the Composite Log GAIC GIMT as a 1-degree of freedom non-directional GIMT. Note that this GIMT is also closely related to the IOS test of [19].

4.2. Methods

4.2.1. Simulated Data Generating Processes

The level and power performance of the six GIMTs are tested using simulation methods described in [23]. First, five data samples, consisting of 1000, 2000, 4000, 8000, and 16,000 exemplars respectively, were created by random sampling a value $x_1$ from a uniform density on the interval [−1, 1] and sampling a value of $x_2$ from a binomial density. A response variable for each exemplar was randomly generated from the predictor $x_1$ using the “true” data generating process specified by the logistic regression model:

$$\log \left(\frac{p\left(y=1\right)}{p\left(y=0\right)}\right)={\beta }_0+{\beta }_1{x}_1+{\beta }_2{x_1}^2+{\beta }_3{x_1}^3,$$

(5)

defined by the true coefficient values: ${\beta }_0=-1.98,{\beta }_1=4.03,{\beta }_2=1.73,{\beta }_3=1.15$.

The response variable y is assigned to a value of one if the computed probability is greater than 0.5, and zero otherwise. Note that the four-parameter regression model in (5) is thus called the correctly specified model, and is used to re-estimate the true coefficient values using the “true” data generating process that is generated using (5).

We also modeled the same binary response variable in the simulated datasets using an “incorrectly” specified model specified by Equation (6):

$$\log \left(\frac{p\left(y=1\right)}{p\left(y=0\right)}\right)={\beta }_0+{\beta }_1{x_1}^3+{\beta }_2\sqrt{\left|x_1\right|}+{\beta }_3{x}_2.$$

(6)

Notice that the parametric forms for the correctly (Equation (5)) and incorrectly (Equation (6)) specified models are the same, except that the incorrectly specified model (Equation (6)) omits $x_1$ and $x_1^2$, and includes an “irrelevant predictor”, $x_2$, and an incorrect transformation, $\sqrt{\left|x_1\right|}$.

Assume a large dataset is constructed by sampling from the data generating process specified by the model in (5). In the correctly specified case when the parameters of the model in (5) are estimated using the dataset generated by the model in (5), the resulting estimators for ${\widehat{A}}_n$ and ${\widehat{B}}_n$ are very similar in magnitude, indicating a lack of evidence of misspecification. On the other hand, in the misspecified case when the parameters of the model in (6) are estimated using the dataset generated by the model in (5), the resulting estimators for ${\widehat{A}}_n$ and ${\widehat{B}}_n$ are quite different, evidencing misspecification (see Theorem 4 of this paper).

In practice, researchers often choose the model that best fits the observed data using in-sample (training data) and out-of-sample (test data) log-likelihood based measures. Two models, however, can have equivalent fits to the observed data using either in-sample ($2n{\widehat{l}}_n$) or out-of-sample (GAIC) model fit measures, yet one of the models can be correctly specified while the other model is not. The data generating process and models used in the simulation studies described here are designed to illustrate this important situation.

The model in (5) when fitted to the dataset generated by (5) had approximately the same in-sample fit ($2n{\widehat{l}}_n=9295.69$) as the in-sample fit ($2n{\widehat{l}}_n=9295.94$) obtained when model (6) was fitted to the dataset generated by (5). Further, the Discrepancy Risk Model Selection Test [38,39,40,41,42,43] did not show a significant difference in the model fits for the models in (5) and (6) (Z = 0.003, p = 0.997).

In addition, using the GAIC [35,36,44] which estimates the out-sample (test data) model fit, the model in (5) when fitted to the dataset generated by (5) had approximately the same out-of-sample fit (GAIC = 9303.6) as the out-sample fit (GAIC = 9305.6) of model (6) to the data set generated by (5). The Discrepancy Risk Model Selection Test [38,39,40,41,42,43] showed no significant difference in GAIC model fits (Z = 0.028, p = 0.98). Thus, despite the presence of model misspecification, both the misspecified model and the correctly specified model provide observationally equivalent fits to the observed data, underscoring the importance of checking for model misspecification.

4.2.2. Estimation of Type 1 and Type 2 Error Rates

To evaluate the level and power performance of the six GIMTs, we estimate the percentage of times that each GIMT incorrectly rejected the null hypothesis in the correctly specified case (or GIMT level), and correctly rejected the null hypothesis in the misspecified case (GIMT power). Since the data were simulated from a known data generating process, the computation of these statistics is straightforward.

Throughout these simulation studies, a MLE was defined as a set of parameter values such that the sup norm of the gradient of the negative average log-likelihood evaluated at the MLE was less than 1 × 10⁻⁸. Further, we avoided fitting models to degenerate simulated data by omitting samples with condition numbers greater than 4.5 × 10¹⁴ to insure numerical stability. The condition number is defined as the maximum eigenvalue divided by the minimum eigenvalue of the inverse of the Hessian covariance matrix estimator. Each simulation was run until m = 10,000 simulated data samples of size n was reached. The sample sizes n for the simulated data represented 6.25%, 12.5%, 25%, 50%, and 100% of the original 16,000-member sample.

4.3. Results and Discussion

4.3.1. Type 1 Error Performance

Table 1. Type 1 error performance of GIMTs using the analytic third derivative formula for pre-specified (nominal) significance levels: 0.01, 0.025, 0.05, and 0.10. Level performance for the directional GIMTs was better than level performance for the non-directional GIMTs. Bootstrap simulation standard errors are shown in parentheses. Computed values are for 10,000 simulated data samples for sample size n = 16,000. df = degrees of freedom.

Generalized Information Matrix Test (GIMT)	Test Type	p = 0.01	p = 0.025	p = 0.05	p = 0.10
Adjusted Classical (≤10 df)	Directional	0.0136	0.0308	0.0550	0.1059
		(0.0012)	(0.0017)	(0.0023)	(0.0031)
Composite GAIC (2 df)	Non-Directional	0.0830	0.1014	0.1225	0.1546
		(0.0027)	(0.0030)	(0.0032)	(0.0036)
Composite Log GAIC (2 df)	Non-Directional	0.0564	0.0742	0.0930	0.1219
		(0.0023)	(0.0026)	(0.0029)	(0.0032)
Fisher Spectra (4 df)	Directional	0.0205	0.0337	0.0584	0.1035
		(0.0014)	(0.0018)	(0.0023)	(0.0030)
Robust Log GAIC (1 df)	Directional	0.0185	0.0360	0.0618	0.1144
		(0.0013)	(0.0018)	(0.0024)	(0.0031)
Robust Log GAIC Ratio (1 df)	Directional	0.0158	0.0335	0.0590	0.1135
		(0.0012)	(0.0018)	(0.0023)	(0.0031)

and

Table 2. Type 1 error performance of GIMTs using the Lancaster-Chesher third derivative approximation for pre-specified (nominal) significance levels: 0.01, 0.025, 0.05, and 0.10. Like the third derivative method in Table 1, level performance for the directional GIMTs was better than level performance for the non-directional GIMTs. Further, for non-directional GIMTs, level performance of the Lancaster-Chesher third derivative approximation for the non-directional GIMTs was better than using third derivative GIMTs. Bootstrap simulation standard errors are shown in parentheses. Computed values are for 10,000 simulated data samples for sample size n = 16,000. df = degrees of freedom.

Generalized Information Matrix Test (GIMT)	Test Type	p = 0.01	p = 0.025	p = 0.05	p = 0.10
Adjusted Classical (≤10 df)	Directional	0.0085	0.0195	0.0409	0.0916
		(0.0009)	(0.0014)	(0.0020)	(0.0029)
Composite GAIC (2 df)	Non-Directional	0.0662	0.0821	0.1006	0.1259
		(0.0024)	(0.0026)	(0.0029)	(0.0032)
Composite Log GAIC (2 df)	Non-Directional	0.0403	0.0498	0.0646	0.0884
		(0.0019)	(0.0021)	(0.0023)	(0.0027)
Fisher Spectra (4 df)	Directional	0.0071	0.0161	0.0264	0.0535
		(0.0008)	(0.0012)	(0.0015)	(0.0021)
Robust Log GAIC (1 df)	Directional	0.0045	0.0138	0.0236	0.0622
		(0.0006)	(0.0011)	(0.0014)	(0.0023)
Robust Log GAIC Ratio (1 df)	Directional	0.0032	0.0097	0.0285	0.0588
		(0.0005)	(0.0009)	(0.0016)	(0.0022)

provide estimated Type 1 errors (i.e., estimated p-values using Theorem 7 and Equation (2)) computed using 10,000 simulated data samples for a sample size of n = 16,000. Empirical level (observed Type 1 error rates) are for pre-specified (nominal) significance levels: 0.01, 0.025, 0.05, and 0.10. The average number of times the null hypothesis was incorrectly rejected by a GIMT in a simulation run was used to estimate the Type 1 error rate. The standard error of the number of times the null hypothesis was incorrectly rejected was defined as the bootstrap sampling error. The average number of times the null hypothesis was incorrectly accepted by a GIMT in a simulation run was used to estimate the Type 2 error rate.

The p-values estimated in Table 1 are based upon the exact formula for the GIMT test statistic provided in Equation (2), which uses the third derivative of the log-likelihood function. Table 2 provides estimates of the Type 1 error rate using formulas that do not require the use of third derivatives of the log-likelihood function by using the Lancaster-Chesher third derivative approximation (see Theorem 8) for the Hessian covariance matrix estimator obtained by substituting the formula $\ddot{\mathsf{\nabla }}{\widehat{d}}_n$ as defined in Equations (3) and (4) for $\mathsf{\nabla }{\widehat{d}}_n$ in Equation (2).

Level performance in Table 1 and Table 2 was evaluated using the Mean Absolute Deviation (MAD), which is defined as the average absolute deviation between an estimated p-value and its theoretical expected asymptotic value. Directional GIMTs showed better performance (MAD = 0.013) than non-directional GIMTs (MAD = 0.44). In addition, the Lancaster-Chesher third derivative approximation method (Table 2) showed better performance (MAD = 0.034) than the analytic third derivative method (Table 1) (MAD = 0.055) for non-directional GIMTs. Level performance for directional GIMTs derived using the Lancaster-Chesher third derivative approximation method (MAD = 0.017) were comparable to directional GIMTs derived using the analytic third derivative method (MAD = 0.0084).

The improved Type 1 error estimation performance of the directional GIMTs may be due to the fact that the directional GIMT statistics had fewer degrees of freedom and thus reduced variance. One possible explanation for the good level performance of the Lancaster-Chesher third derivative approximation method is that this method uses assumptions that hold under the null hypothesis to derive an alternative GIMT covariance matrix estimator without calculating third derivatives. In simulation studies where the null hypothesis of correct model specification holds, key large sample assumptions of the Lancaster-Chesher third derivative approximation method are satisfied by construction. This suggests that, in some cases, for the purpose of estimating Type 1 errors, the Lancaster-Chesher method may be appropriate for large sample sizes. On the other hand, Taylor [18] has provided examples where the size properties of the Lancaster-Chesher method are poor.

4.3.2. Level-Power Analyses

The level-power performance of the new GIMTs were investigated by examining how the estimated Type 1 and Type 2 errors varied as a function of test significance level. In particular, for a range of possible significance levels, the estimated power (i.e., percent correct rejections) and estimated Type 1 error (i.e., percent incorrect rejections) can be calculated to obtain a Receiver Operating Characteristic (ROC) curve [14,45,46,47]. The Area under the ROC (AUROC) is measure of discrimination performance. An AUROC = 1.0 indicates perfect discrimination performance and an AUROC = 0.5 indicates chance discrimination performance [45,46,47]. Although discrimination performance can vary dramatically as a function of test problem difficulty, this paradigm is useful for comparing discrimination performance of different GIMT statistics with respect to a particular test problem.

Figure 1 shows the Level-power for GIMTs using the analytic 3rd derivative for the inverse Hessian matrix estimator by sample size. With respect to the chosen test problem described in the text, these GIMTs obtain nearly perfect performance in correct rejection of the null hypothesis and correct acceptance of the null hypothesis when the sample size in this simulation study exceeds 4000 exemplars.

Figure 2 shows the Level-power for GIMTs using the Lancaster-Chesher 3rd derivative approximation. With respect to the chosen test problem these GIMTs obtain excellent performance in correct rejection of the null hypothesis and correct acceptance of the null hypothesis when the sample size in this simulation study is near 16,000 exemplars. However, while the Adjusted Classical GIMT evidences excellent performance across sample sizes in all cases, the other GIMTs show poor Level-Power performance below 15,000 exemplars with the Lancaster-Chesher 3rd derivative approximation. In addition, with the exception of the Adjusted Classical GIMT, there is not a clear difference in performance between the directional and non-directional tests. These results are consistent with the observations of previous investigators regarding the power performance of the Lancaster-Chesher method (e.g., [14,15,17,18]).

5. Conclusions

This paper formally introduces a unified framework for specification testing that is applicable to a wide range of smooth probability models including, for example, the class of generalized linear models (e.g., [48,49,50]), linear and nonlinear regression (e.g., [51,52]), structural equation models with or without latent variables (e.g., [53,54]), and hierarchical linear models (e.g., [55]). The essential idea is based upon the Contrapositive of the Information Matrix Equality (Theorem 4), which asserts that observed differences between the inverse Hessian covariance matrix estimator ${\widehat{A}}_n$ and the inverse OPG covariance matrix estimator ${\widehat{B}}_n$ are indicators of the presence of model misspecification.

Theorem 6 provided explicit conditions for ensuring that ${\widehat{s}}_n$ converges with probability one to $s\left(A^{\ast },B^{\ast }\right)$ as $n\to \infty$. Theorem 7 provided explicit conditions for showing that if the null hypothesis $H_0:s\left(A^{\ast },B^{\ast }\right)=0_r$ holds, then a Wald test statistic can be constructed that has an asymptotic chi-squared distribution with r degrees of freedom. If, however, the null hypothesis $H_0:s\left(A^{\ast },B^{\ast }\right)=0_r$ is false, then that same Wald test statistic asymptotically converges to infinity with probability one. Proposition 1 asserts that: (1) if the probability model is correctly specified then $H_0:s\left(A^{\ast },B^{\ast }\right)=0_r$, and (2) if $H_0:s\left(A^{\ast },B^{\ast }\right)=0_r$ is false then the probability model is misspecified.

In the simulation studies, each of the new directional and non-directional GIMTs exhibited excellent level-power performance using the third derivative formulas for the GIMT covariance matrix estimator. However, performance in estimating the Type 1 error rate varied for different GIMTs, indicating the importance of simulation studies for characterizing the performance of new GIMTs derived within the GIMT framework. In fact, the performance of the directional GIMTs was better than the non-directional GIMTs. The simulation studies also showed that the level-power performance of the GIMTs declined with smaller sample sizes for the Lancaster-Chesher third derivative approximation formula. In addition, the appealing level-power performance of the Adjusted Classical GIMT for both the true third derivative and Lancaster-Chesher third derivative approximation suggests that additional research into the development of GIMTs with adjusted covariance matrices as described in Proposition 2 is merited. It is also important to emphasize that the alternative model used in the above power analyses was chosen such that its fit to the observed data was comparable to the fit of the “true” model that generated the data.

In summary, the simulation studies illustrate a general methodology for using the GIMT framework to derive and evaluate new model misspecification tests. We showed that it is possible for an incorrectly specified model to appear to fit the data well, while testing positive for model misspecification (i.e., reject the null hypothesis that the model is correctly specified). To reach proper statistical inferences when interpreting estimates of the parameters to a fitted model, it is critical to consider both model fit and model specification.

In conclusion, a unified GIMT framework has been presented for identifying, classifying, and developing information matrix type statistical tests for the detection of model misspecification for smooth finite-dimensional probability models. This GIMT framework provides a practical and powerful methodology for the development of both directional and non-directional GIMTs for a wide range of smooth probability models. Furthermore, unlike some existing methods for specification testing in logistic regression modeling, the degrees of freedom of the GIMT test statistic do not increase as a function of the number of distinct patterns of predictor variable values, suggesting that GIMTs will have good level and power performance [51,56,57,58]. In the real world, it is inevitable that model misspecification will manifest itself in different ways for different probability models and in different situations. Accordingly, it is desirable to have a variety of tests for assessing model misspecification as some tests may be more appropriate than others in detecting the presence of model misspecification in different situations.