Generalized Empirical Likelihood-Based Focused Information Criterion and Model Averaging

Abstract

This paper develops model selection and averaging methods for moment restriction models. We first propose a focused information criterion based on the generalized empirical likelihood estimator. We address the issue of selecting an optimal model, rather than a correct model, for estimating a specific parameter of interest. Then, this study investigates a generalized empirical likelihood-based model averaging estimator that minimizes the asymptotic mean squared error. A simulation study suggests that our averaging estimator can be a useful alternative to existing post-selection estimators.

1. Introduction

This paper develops model selection and averaging methods for moment restriction models. We first propose a focused information criterion (FIC) based on the generalized empirical likelihood (GEL) estimator [1,2], which nests the empirical likelihood (EL) [3,4] and exponential tilting (ET) [5,6] estimators as special cases. Motivated by Claeskens and Hjort [7], we address the issue of selecting an optimal model for estimating a specific parameter of interest, rather than identifying a correct model or selecting a model with good global fit. Then, as an extension of FIC, this study presents a GEL-based frequentist model averaging (FMA) estimator that is designed to minimize the mean squared error (MSE) of the estimator.

Traditional model selection methods, such as the Akaike information criterion (AIC) and the Bayesian information criterion (BIC), select a single model regardless of the specific goal of inference [8,9]. AIC selects a model that is close to the true data generating process (DGP) in terms of Kullback-Leibler discrepancy, while BIC selects the model with the highest posterior probability. However, a model with good global fit is not necessarily a good model for estimating a specific parameter. For instance, Hansen [10] considers the problem of deciding the order of autoregressive models. His simulation study demonstrates that the AIC-selected model does not necessarily produce a good estimate of the impulse response. This result reveals that the best model generally differs for different intended uses of the model.

In their seminal work, Claeskens and Hjort [7] established an FIC that is designed to select the optimal model depending on its intended use. Their goal is to select the model that attains the minimum MSE of the maximum likelihood estimator for the parameter of interest, which they call the focus parameter. The FIC is constructed from an asymptotic estimate of the MSE.

Since then, an FIC has been derived for several models. Claeskens, Croux and Kerckhoven [11] proposed an FIC for logistic regressions. Hjort and Claeskens [12] proposed an FIC for the Cox hazard regression model. Zhang and Liang [13] developed an FIC for the generalized additive partial linear model. Models studied in those papers are likelihood-based. However, econometric models are often specified via moment restrictions rather than parametric density functions. This paper indicates that the idea of Claeskens and Hjort [7] is applicable to moment restriction models. Our FIC is constructed using an asymptotic estimate of the MSE of the GEL estimator.

Model selection for moment restriction models is still underdeveloped. Andrews and Lu [14] proposed selection criteria based on the J-statistic of the generalized method of moments (GMM) estimator [15]. Hong, Preston and Shum [16] extended the results of Andrews and Lu to the GEL estimation. Sueishi [17] developed information criteria similar to the AIC. The goal of Andrews and Lu [14] and Hong, Preston and Shum [16] was to identify the correct model, whereas Sueishi [17] selects the best approximating model in terms of Cressie-Read discrepancy. Although these criteria are useful in many applications, they do not address the issue of selecting the model that best serves its intended purpose.

Model averaging is an alternative to model selection. Inference after model selection is typically conducted as if the selected model is the true DGP. However, this ignores uncertainty introduced by model selection. Rather than conditioning on the single selected model, the averaging technique uses all candidate models to incorporate model selection uncertainty. Although Bayesian methods are predominant in the literature [18], there is also a growing FMA literature for likelihood-based models [19,20,21]. See also Yang [22], Leung and Barron [23] and Goldenshluger [24] for related issues.

In the FMA literature, it is often of particular interest to obtain an optimal averaging estimator in terms of a certain loss [25,26,27,28]. This study investigates a GEL-based averaging method that minimizes the asymptotic mean squared error in a framework similar to that of Hjort and Claeskens [21]. A simulation study indicates that our averaging estimator outperforms existing post-model-selection estimators.

Although this study investigates GEL-based methods, in general, its results are readily applied to the two-step GMM estimator, because our results rely only on first-order asymptotic theory. However, the two-step GMM estimator often suffers from a large bias that cannot be captured by first-order asymptotics, even if the model is correctly specified. Because the FIC addresses a trade-off between misspecification bias and estimation variance, the GEL estimator will be more suitable for our framework.

Now, we review related works. DiTraglia [29] proposes an instrument selection criterion for GMM that is based on the concept of FIC. Our approach resembles DiTraglia’s, but his interest is instrument selection, whereas ours is model selection. DiTraglia intentionally uses an invalid large set of instruments to improve efficiency; we intentionally use a wrong small model to improve efficiency. Liu [30] proposes an averaging estimator for the linear regression model by using a local asymptotic framework. Although Liu considers exogenous regressors, we allow endogenous regressors. Martins and Gabriel [31] consider GMM-based model averaging estimators under a framework different from ours.

The remainder of the paper is organized as follows. Section 2 describes our local misspecification framework. Section 3 derives the FIC. Section 4 discusses the FMA estimator. Section 5 provides a simple example for which our methods are applicable. Section 6 presents the result of Monte Carlo study. Section 7 concludes.

2. Local Misspecification Framework

We first introduce our setup. The basic construct follows Claeskens and Hjort [7]. There is a smallest and a largest model in our set of candidate models. The smallest, which we call the reduced model, has a p dimensional unknown parameter vector, $\theta ={({\theta }_1,\cdots ,{\theta }_p)}^{\prime }$. The largest, or the full model, has an additional q dimensional unknown parameter vector, $\gamma ={({\gamma }_1,\cdots ,{\gamma }_q)}^{\prime }$. The full model is assumed to be correctly specified and nests the reduced model; i.e., the reduced model corresponds to the special case of the full model in which $\gamma ={\gamma }_0={({\gamma }_{0,1},\cdots ,{\gamma }_{0,q})}^{\prime }$ for some known ${\gamma }_0$. Typically, ${\gamma }_0$ is a vector of zeros: ${\gamma }_0={(0,\cdots ,0)}^{\prime }$. An example is given in Section 5.

There are up to $2^q$ submodels, all of which have $\theta$ as the common parameter vector. A submodel treats some elements of $\gamma$ as unknown parameters and is indexed by a subset, S, of $\{1,\cdots ,q\}$. The model, S, contains parameters, ${\gamma }_j$, such that $j\in S$. Thus, the reduced and full models correspond to $S=\varphi$ and $S=\{1,\cdots ,q\}$, respectively. We use “red” and “full” to denote the reduced and full models, respectively.

The focus parameter, $\mu$, which is the parameter of interest, is a function of $\theta$ and $\gamma$: $\mu =\mu (\theta ,\gamma )$. It could be merely an element of $\theta$. Prior knowledge or economic theories suggest that $\theta$ should be estimated, but we are unsure which elements of $\gamma$ should be treated as unknown parameters. Estimating a larger model usually implies a lesser modeling bias and a larger estimation variance. However, if the reduced model is globally misspecified in the sense that the violation of the moment restriction does not disappear even in the limit, then the misspecification bias asymptotically dominates the variance of the GEL estimator. Thus, we cannot make a reasonable comparison of bias and variance in the asymptotic framework.

A local misspecification framework is introduced to take into account the bias-variance trade-off. Let $y_1,\cdots ,y_n$ be i.i.d.random vectors from an unknown density, $f_n\left(y\right)$, which depends on the sample size, n.1The functional form of $f_n\left(y\right)$ is not specified. The full model is defined via the following moment restriction:

$$\begin{array}{c}\hfill E_n\left[m(y_i,{\theta }_0,{\gamma }_0+\delta /\sqrt{n})\right]\equiv \int m(y,{\theta }_0,{\gamma }_0+\delta /\sqrt{n})f_n\left(y\right)dy=0,\end{array}$$

(1)

where $m:{\mathbb{R}}^{d_y}\times \Theta \times \Gamma \to {\mathbb{R}}^l$ is a known vector-valued function up to the parameters. For each n, the true parameter values of $\theta$ and $\gamma$ are ${\theta }_0$ and ${\gamma }_0+\delta /\sqrt{n}$, respectively. Note that ${\gamma }_0$ is known, but ${\theta }_0$ and $\delta$ are unknown. We assume that $l>p+q$; i.e., the model is over-identified.

The moment function of the reduced model is $m(y,\theta ,{\gamma }_0)$. The reduced model is misspecified in the sense that there is no value ${\theta }^{*}\in \Theta$, such as $E_n\left[m(y_i,{\theta }^{*},{\gamma }_0)\right]=0$, for any fixed n. However, if the moment function is differentiable with respect to $\gamma$, then (1) implies that the reduced model satisfies:

$$\begin{array}{c}\hfill ∥E_n\left[m(y_i,{\theta }_0,{\gamma }_0)\right]∥=∥E_n\left[\frac{\partial m(y_i,{\theta }_0,\overline{\gamma })}{\partial {\gamma }^{\prime }}\right]\delta /\sqrt{n}∥=O(1/\sqrt{n})\end{array}$$

for some vector, $\overline{\gamma }$ between ${\gamma }_0$ and ${\gamma }_0+\delta /\sqrt{n}$. Thus, even though the moment restriction is invalid at $({\theta }_0,{\gamma }_0)$, the violation disappears in the limit. A similar relationship also holds for the other submodels. As the next section reveals, under this framework, the squared bias and variance of the GEL estimator are both of the order, $O(1/n)$. Hence, the trade-off between bias and variance can be considered. If $\delta$ is sufficiently small, it might be better to set $\gamma ={\gamma }_0$ rather than estimate $\gamma$.

In general, the dimension of the moment function can differ among submodels. For instance, consider a linear instrumental variable model. The model (structural form) can be estimated as long as the number of instruments exceeds or equals the number of unknown parameters. Thus, it is possible to use only a subset of instruments to estimate a submodel. For ease of exposition, however, we consider only the case where the dimension of the moment function is fixed for all submodels.

3. Focused Information Criterion

To construct an FIC, we first derive the asymptotic distribution of the GEL estimator under the local misspecification framework. Newey [32] and Hall [33] obtained a similar result in the case of GMM estimation to analyze the local power properties of specification tests.

A model, S, contains $p+q_S$ unknown parameters. The moment function of the model is denoted as $m(y,\theta ,{\gamma }_S)=m(y,\theta ,{\gamma }_S,{\gamma }_{0,S^C})$, where $S^C$ is the complementary set of S. The values of ${\gamma }_j$ are set to be their null values ${\gamma }_{0,j}$ for $j\in S^C$.

Let $\rho \left(v\right)$ be a concave function on its domain, $\mathcal{V}$, which is an open interval containing zero. We normalize $\rho \left(v\right)$, so that ${\rho }_1\left(0\right)={\rho }_2\left(0\right)=-1$, where ${\rho }_j\left(v\right)=d^j\rho \left(v\right)/d{v}^j$. The GEL estimator of $(\theta ,{\gamma }_S)$ is obtained by solving the saddle-point problem:

$$\begin{array}{c}\hfill ({\widehat{\theta }}_S,{\widehat{\gamma }}_S)=arg\underset{\theta \in \Theta ,{\gamma }_S\in {\Gamma }_S}{min}\underset{\tau \in \mathcal{T}}{max}\frac{1}{n}\sum _{i=1}^n\rho \left({\tau }^{\prime }m(y_i,\theta ,{\gamma }_S)\right),\end{array}$$

where ${\Gamma }_S\subset {\mathbb{R}}^{q_S}$ is the parameter space of ${\gamma }_S$ and $\mathcal{T}\subset {\mathbb{R}}^l$ is the set of feasible values of $\tau$. The EL and ET estimators are special cases with $\rho \left(v\right)=log(1-v)$ and $\rho \left(v\right)=-exp\left(v\right)$, respectively. Although ${\widehat{\theta }}_S$ has p elements for any S, we adopt the subscript, S, to emphasize that the value of the estimator depends on S.

Let $m_i=m(y_i,{\theta }_0,{\gamma }_0)$, $m_{\theta i}={\left.\frac{\partial m(y_i,\theta ,\gamma )}{\partial {\theta }^{\prime }}\right|}_{\theta ={\theta }_0,\gamma ={\gamma }_0}$, and $m_{\gamma i}={\left.\frac{\partial m(y_i,\theta ,\gamma )}{\partial {\gamma }^{\prime }}\right|}_{\theta ={\theta }_0,\gamma ={\gamma }_0}$. Furthermore, let $m_{{\gamma }_{S}i}={\left.\frac{\partial m(y_i,\theta ,\gamma )}{\partial {\gamma }_S^{\prime }}\right|}_{\theta ={\theta }_0,\gamma ={\gamma }_0}$. We define:

$$\begin{array}{c}\hfill J_S=\left(\begin{array}{cc}{J}_{00}& J_{01,S}\\ J_{10,S}& J_{11,S}\end{array}\right)=\left(\begin{array}{cc}E{\left[m_{\theta i}\right]}^{\prime }E{\left[m_i{m}_i^{\prime }\right]}^{-1}E\left[m_{\theta i}\right]& E{\left[m_{{\theta }_i}\right]}^{\prime }E{\left[m_i{m}_i^{\prime }\right]}^{-1}E\left[m_{{\gamma }_{S}i}\right]\\ E{\left[m_{{\gamma }_{S}i}\right]}^{\prime }E{\left[m_i{m}_i^{\prime }\right]}^{-1}E\left[m_{\theta i}\right]& E{\left[m_{{\gamma }_{S}i}\right]}^{\prime }E{\left[m_i{m}_i^{\prime }\right]}^{-1}E\left[m_{{\gamma }_{S}i}\right]\end{array}\right),\end{array}$$

where E denotes the expectation with respect to $f\left(y\right)\equiv {lim}_{n\to \infty }{f}_n\left(y\right)$. It is assumed that $f\left(y\right)$ satisfies:

$$\begin{array}{c}\hfill E\left[m_i\right]=\int m(y,{\theta }_0,{\gamma }_0)f\left(y\right)dy=0.\end{array}$$

For the full model, we denote:

$$\begin{array}{c}\hfill J_{\mathrm{full}}=\left(\begin{array}{cc}{J}_{00}& J_{01}\\ J_{10}& J_{11}\end{array}\right).\end{array}$$

Then, we can write $J_{01,S}=J_{01}{\pi }_S^{\prime }$ and $J_{11,S}={\pi }_S{J}_{11}{\pi }_S^{\prime }$, where ${\pi }_S$ is the projection matrix of size, $q_S\times q$, that maps $\gamma$ to the subvector, ${\gamma }_S$: ${\pi }_S\gamma ={\gamma }_S$.

Let ${\widehat{Q}}_n(\theta ,\gamma ,\tau )=n^{-1}{\sum }_{i=1}^n\rho \left({\tau }^{\prime }m(y_i,\theta ,\gamma )\right)$ and $Q_n(\theta ,\gamma ,\tau )=E_n\left[\rho \left({\tau }^{\prime }m(y_i,\theta ,\gamma )\right)\right]$. Furthermore, let $Q(\theta ,\gamma ,\tau )=E\left[\rho \left({\tau }^{\prime }m(y_i,\theta ,\gamma )\right)\right]$. To obtain the asymptotic distribution of the GEL estimator, we impose the following conditions:

Assumption 3.1

1. 

$\Theta \subset {\mathbb{R}}^p$, $\Gamma \subset {\mathbb{R}}^q$, and $\mathcal{T}\subset {\mathbb{R}}^l$ are compact.

2. 

$m(y,\theta ,\gamma )$ is continuous in $\theta \in \Theta$ and $\gamma \in \Gamma$ for almost every y.

3. 

${sup}_{\theta \in \Theta ,\gamma \in \Gamma ,\tau \in \mathcal{T}}\left|{\widehat{Q}}_n(\theta ,\gamma ,\tau )-Q_n(\theta ,\gamma ,\tau )\right|\stackrel{p}{\to }0$ under the sequence of $f_n\left(y\right)$.

4. 

$\left|Q_n(\theta ,\gamma ,\tau )-Q(\theta ,\gamma ,\tau )\right|\to 0$ as $n\to \infty$ for all $\theta \in \Theta$, $\gamma \in \Gamma$, and $\tau \in \mathcal{T}$.

5. 

$E\left[m(y_i,\theta ,\gamma )m{(y_i,\theta ,\gamma )}^{\prime }\right]$ is nonsingular for all $\theta \in \Theta$ and $\gamma \in \Gamma$.

6. 

$({\theta }_0,{\gamma }_0)$ is the unique solution to $E\left[m(y_i,\theta ,\gamma )\right]=0$ and $({\theta }_0,{\gamma }_0)\in \mathit{int}(\Theta \times \Gamma )$.

7. 

$\rho \left(v\right)$ is twice continuously differentiable in a neighborhood of zero.

8. 

$E\left[m_{\theta i}\right]$ and $E\left[m_{\gamma i}\right]$ are of full rank.

9. 

${sup}_n{E}_n[\parallel m_i{\parallel }^{2+\alpha }]<\infty$ for some $\alpha >0$.

10. 

$m(y,\theta ,\gamma )$ is continuously differentiable in $\theta$ and $\gamma$ in a neighborhood, $\mathcal{N}$, of $({\theta }_0,{\gamma }_0)$.

11. 

${sup}_{\theta ,\gamma \in \mathcal{N}}\left|n^{-1}{\sum }_{i=1}^n\frac{\partial m(y_i,\theta ,\gamma )}{\partial {\theta }^{\prime }}-E_n\left[\frac{\partial m(y_i,\theta ,\gamma )}{\partial {\theta }^{\prime }}\right]\right|\stackrel{p}{\to }0$ and ${sup}_{\theta ,\gamma \in \mathcal{N}}\left|n^{-1}{\sum }_{i=1}^n\frac{\partial m(y_i,\theta ,\gamma )}{\partial {\gamma }^{\prime }}-E_n\left[\frac{\partial m(y_i,\theta ,\gamma )}{\partial {\gamma }^{\prime }}\right]\right|\stackrel{p}{\to }0$ under the sequence of $f_n\left(y\right)$.

12. 

$\parallel E_n\left[m_{\theta i}\right]-E\left[m_{\theta i}\right]\parallel \to 0$ and $\parallel E_n\left[m_{\gamma i}\right]-E\left[m_{\gamma i}\right]\parallel \to 0$ as $n\to \infty$.

13. 

$\parallel E_n\left[m_i{m}_i^{\prime }\right]-E\left[m_i{m}_i^{\prime }\right]\parallel \to 0$ as $n\to \infty$.

Conditions are rather high-level and strong. Some conditions can be replaced with primitive and weaker conditions [34].

We obtain the following lemma.

Lemma 3.1 Suppose Assumption 3.1 holds. Then, under the sequence of $f_n\left(y\right)$, we have:

$$\begin{array}{c}\hfill \sqrt{n}\left(\begin{array}{c}{\widehat{\theta }}_S-{\theta }_0\\ {\widehat{\gamma }}_S-{\gamma }_{0,S}\end{array}\right)\stackrel{d}{\to }N\left(J_S^{-1}\left(\begin{array}{c}{J}_{01}\\ {\pi }_S{J}_{11}\end{array}\right)\delta ,J_S^{-1}\right).\end{array}$$

The proof is given in the Appendix.

If the model, S, is correctly specified, then the limiting distribution of the GEL estimator is $N(0,J_S^{-1})$. Therefore, as usual, local misspecification affects only the mean of the limiting distribution.

Next, we get the asymptotic distribution of the GEL estimator for the focus parameter. Additional notations are introduced. Let $Q={(J_{11}-J_{10}{J}_{00}^{-1}{J}_{01})}^{-1}$ and $Q_S={\left({\pi }_S{Q}^{-1}{\pi }_S^{\prime }\right)}^{-1}$; i.e., Q and $Q_S$ are the lower right block matrices of $J_{\mathrm{full}}^{-1}$ and $J_S^{-1}$, respectively. Let $G_S={\pi }_S^{\prime }{Q}_S{\pi }_S{Q}^{-1}$. We assume that $\mu (\theta ,\gamma )$ is differentiable with respect to $\theta$ and $\gamma$. Let:

$$\begin{array}{c}\hfill w=J_{10}{J}_{00}^{-1}\frac{\partial \mu }{\partial \theta }-\frac{\partial \mu }{\partial \gamma }\text{and}{\tau }_0^2={\left(\frac{\partial \mu }{\partial \theta }\right)}^{\prime }{J}_{00}^{-1}\frac{\partial \mu }{\partial \theta },\end{array}$$

where the partial derivatives are evaluated at $({\theta }_0,{\gamma }_0)$. The true focus parameter is denoted as ${\mu }_{\text{true}}=\mu ({\theta }_0,{\gamma }_0+\delta /\sqrt{n})$. Moreover, the GEL estimator of $\mu$ for the model, S, is denoted as ${\widehat{\mu }}_S=\mu ({\widehat{\theta }}_S,{\widehat{\gamma }}_S,{\gamma }_{0,S^C})$. Lemma 3.1 and the delta method imply the following theorem:

Theorem 3.1 Suppose Assumption 3.1 holds. Then, under the sequence of $f_n\left(y\right)$, we have:

$$\begin{array}{c}\hfill D_n\equiv \sqrt{n}({\widehat{\gamma }}_{\mathrm{full}}-{\gamma }_0)\stackrel{d}{\to }D\sim N(\delta ,Q)\end{array}$$

and:

$$\begin{array}{c}\hfill \sqrt{n}\left({\widehat{\mu }}_S-{\mu }_{\mathrm{true}}\right)\stackrel{d}{\to }{\Lambda }_S\equiv {\Lambda }_0+w^{\prime }(\delta -G_{S}D),\end{array}$$

where ${\Lambda }_0\sim N(0,{\tau }_0^2)$ is independent of D.

The proof is almost the same as that of Lemma 3.3 in Hjort and Claeskens [21], so it is omitted.

Because $G_{\mathrm{full}}=I$ and $G_{\mathrm{red}}=0$, as the special cases of the theorem, we have:

$$\begin{array}{ccc}& & \sqrt{n}\left({\widehat{\mu }}_{\mathrm{full}}-{\mu }_{\mathrm{true}}\right)\stackrel{d}{\to }N(0,{\tau }_0^2+w^{\prime }Qw),\hfill \\ & & \sqrt{n}\left({\widehat{\mu }}_{\mathrm{red}}-{\mu }_{\mathrm{true}}\right)\stackrel{d}{\to }N(w^{\prime }\delta ,{\tau }_0^2).\hfill \end{array}$$

Therefore, in terms of the asymptotic MSE, the reduced model is better than the full model if ${\left(w^{\prime }\delta \right)}^2<w^{\prime }Qw$, which is the case when the deviation of the reduced model from the true DGP is small.

More generally, Theorem 3.1 implies that the MSE of the limiting distribution of ${\widehat{\mu }}_S$ is:

$$\begin{array}{c}\hfill \mathrm{mse}(S,\delta )={\tau }_0^2+w^{\prime }{\pi }_S^{\prime }{Q}_S{\pi }_{S}w+w^{\prime }(I-G_S)\delta {\delta }^{\prime }{(I-G_S)}^{\prime }w.\end{array}$$

(2)

The idea behind FIC is to estimate (2) for each model and select the model that attains the minimum estimated MSE.

All components in (2) except $\delta$ can be estimated easily by using their sample analogs. However, a consistent estimator for $\delta$ is unavailable, because $D_n$ converges in distribution to a normal random variable. This difficulty is inevitable, as long as we utilize the local misspecification framework. Because the mean of $D{D}^{\prime }$ is $\delta {\delta }^{\prime }+Q$, following Claeskens and Hjort [7], we use $D_n{D}_n^{\prime }-\widehat{Q}$ to estimate $\delta {\delta }^{\prime }$. Then, the sample counterpart of (2) is:

$$\begin{array}{ccc}\hfill \widehat{\mathrm{mse}}\left(S\right)& =& {\widehat{\tau }}_0^2+{\widehat{w}}^{\prime }{\pi }_S{\widehat{Q}}_S{\pi }_S\widehat{w}+{\widehat{w}}^{\prime }(I-{\widehat{G}}_S)(D_n{D}_n^{\prime }-\widehat{Q}){(I-G_S)}^{\prime }\widehat{w}\hfill \\ & =& {\widehat{w}}^{\prime }(I-{\widehat{G}}_S)D_n{D}_n^{\prime }{(I-{\widehat{G}}_S)}^{\prime }\widehat{w}+2{\widehat{w}}^{\prime }{\pi }_S^{\prime }{\widehat{Q}}_S{\pi }_S\widehat{w}+{\widehat{\tau }}_0^2-{\widehat{w}}^{\prime }\widehat{Q}\widehat{w},\hfill \end{array}$$

which is an asymptotically unbiased estimator for (2). Because the last two terms do not depend on the model, we can ignore them for the purpose of model selection. Let ${\widehat{\psi }}_{\text{full}}={\widehat{w}}^{\prime }{D}_n$ and ${\widehat{\psi }}_S={\widehat{w}}^{\prime }{\widehat{G}}_S{D}_n$. Then, our FIC for the model, S, is:

$$\begin{array}{ccc}\hfill {\mathrm{FIC}}_S& =& {\widehat{w}}^{\prime }(I-{\widehat{G}}_S)D_n{D}_n^{\prime }{(I-{\widehat{G}}_S)}^{\prime }\widehat{w}+2{\widehat{w}}^{\prime }{\pi }_S^{\prime }{\widehat{Q}}_S{\pi }_S\widehat{w}\hfill \\ & =& {({\widehat{\psi }}_{\text{full}}-{\widehat{\psi }}_S)}^2+2{\widehat{w}}_S^{\prime }{\widehat{Q}}_S{\widehat{w}}_S,\hfill \end{array}$$

(3)

where ${\widehat{w}}_S={\pi }_S\widehat{w}$. The bigger the model is, the smaller the first term and the larger the second term in (3). Since w depends on $\mu$, FIC can be used to select an appropriate submodel, depending on the parameter of interest.

Although we consider only the case where $\mu$ is a scalar, our FIC is also applicable to a vector-valued focus parameter by viewing each element of the vector as a different scalar-valued focus parameter. Different models might be used to estimate different elements of the vector.

We conclude this section with a remark on the estimation of $\delta {\delta }^{\prime }$. Because we estimate $\delta {\delta }^{\prime }$ by $D_n{D}_n^{\prime }-\widehat{Q}$, the estimate can be negative definite in finite sample. That means that the squared bias term can be negative. To avoid such cases, as suggested by Claeskens and Hjort [35], we can also use the following bias-corrected FIC:

$$\begin{array}{c}\hfill {\mathrm{FIC}}_S^{*}=\left\{\begin{array}{cc}{\mathrm{FIC}}_S\hfill & \text{if}{N}_n\left(S\right)\text{does}\text{not}\text{take}\text{place}\hfill \\ {\widehat{w}}^{\prime }(I+{\widehat{G}}_S)\widehat{Q}\widehat{w}\hfill & \text{if}{N}_n\left(S\right)\text{takes}\text{place},\hfill \end{array}\right.\end{array}$$

where $N_n\left(S\right)$ is the event of negligible bias:

$$\begin{array}{c}\hfill {\left\{{\widehat{w}}^{\prime }(I-{\widehat{G}}_S){\widehat{\delta }}_{\text{full}}\right\}}^2<{\widehat{w}}^{\prime }\left(\widehat{Q}-{\pi }_S^{\prime }{\widehat{Q}}_S{\pi }_S\right)\widehat{w}.\end{array}$$

See Section 6.4 of Claeskens and Hjort [35] for details.

4. Model Averaging

This section extends the result of Section 3 to the averaging problem. In the FMA literature, it is often of particular interest to obtain an optimal averaging estimator in terms of a certain loss. We consider a possibility of obtaining the best averaging weights that minimize the MSE in the local misspecification framework. A similar analysis is presented in Liu [30] in the case of linear regression.

Let $\mathcal{A}$ be the set of all candidate models. We consider an averaging estimator for the focus parameter of the form:

$$\begin{array}{c}\hfill \widehat{\mu }=\sum _{S\in \mathcal{A}}c\left(S\right){\widehat{\mu }}_S,\end{array}$$

where the weights, $c\left(S\right)$, add up to unity. Note that a post-selection estimator of $\mu$ can also be written in this form. Let $S_{\mathrm{FIC}}$ be the FIC-selected model. Then the post-selection estimator using FIC is:

$$\begin{array}{c}\hfill {\widehat{\mu }}_{\mathrm{FIC}}=\sum _{S\in \mathcal{A}}1(S=S_{\mathrm{FIC}}){\widehat{\mu }}_S,\end{array}$$

where $1(·)$ is the indicator function. Thus, the post-selection estimator is a special case of the averaging estimator.

If the weights are not random, then it is straightforward from Theorem 3.1 that:

$$\begin{array}{c}\hfill \sqrt{n}\left(\widehat{\mu }-{\mu }_{\mathrm{true}}\right)\stackrel{d}{\to }\Lambda \equiv \sum _{S\in \mathcal{A}}c\left(S\right){\Lambda }_S\stackrel{d}{=}{\Lambda }_0+w^{\prime }(\delta -\widehat{\delta }\left(D\right)),\end{array}$$

where $\widehat{\delta }\left(D\right)={\sum }_{S\in \mathcal{A}}c\left(S\right)G_{S}D$. Therefore, the asymptotic mean and variance of the averaging estimator are given by:

$$\begin{array}{ccc}\hfill E[\Lambda ]& =& \sum _{S\in \mathcal{A}}c\left(S\right)w^{\prime }(I-G_S)\delta ,\hfill \\ \hfill \mathrm{Var}[\Lambda ]& =& {\tau }_0^2+\sum _{S,S^{\prime }\in \mathcal{A}}c\left(S\right)c\left(S^{\prime }\right)w^{\prime }\left(G_{S}Q{G}_{S^{\prime }}^{\prime }\right)w.\hfill \end{array}$$

Thus, there is a set of weights that minimizes the asymptotic MSE of $\widehat{\mu }$.

Suppose there are M candidate models: $S_1,\cdots ,S_M$. Let $C={(c\left(S_1\right),\cdots ,c\left(S_M\right))}^{\prime }$ be a vector of averaging weights, which is in the unit simplex in ${\mathbb{R}}^M$:

$$\begin{array}{c}\hfill \mathcal{H}=\left\{C\in {[0,1]}^M:\sum _{i=1}^{M}c\left(S_i\right)=1\right\}.\end{array}$$

Ignoring ${\tau }_0^2$, which does not depend on the model, the optimal weight vector, $C^{*}$, that minimizes the asymptotic MSE is:

$$\begin{array}{c}\hfill C^{*}=arg\underset{C\in \mathcal{H}}{min}{C}^{\prime }AC,\end{array}$$

where A is an $M\times M$ matrix, whose $(i,j)$ element is given by:

$$\begin{array}{c}\hfill A_{\left[ij\right]}=w^{\prime }(I-G_{S_i})\delta {\delta }^{\prime }{(I-G_{S_j})}^{\prime }w+w^{\prime }\left(G_{S_i}Q{G}_{S_j}^{\prime }\right)w.\end{array}$$

If we replace A with its appropriate estimate, $\widehat{A}$, we obtain a feasible estimator:

$$\begin{array}{c}\hfill \widehat{C}=arg\underset{C\in \mathcal{H}}{min}{C}^{\prime }\widehat{A}C.\end{array}$$

(4)

For instance, if we estimate $\delta {\delta }^{\prime }$ by $D_n{D}_n^{\prime }-\widehat{Q}$, then:

$$\begin{array}{c}\hfill {\widehat{A}}_{\left[ij\right]}={\widehat{w}}^{\prime }(I-{\widehat{G}}_{S_i})(D_n{D}_n^{\prime }-\widehat{Q}){(I-{\widehat{G}}_{S_j})}^{\prime }\widehat{w}+{\widehat{w}}^{\prime }\left({\widehat{G}}_{S_i}\widehat{Q}{\widehat{G}}_{S_j}^{\prime }\right)\widehat{w}.\end{array}$$

Although there is no closed-form solution for (4), it can be solved numerically by a usual quadratic programing algorithm.

Unfortunately, $\widehat{C}$ cannot be a consistent estimator for $C^{*}$, because there is no consistent estimator for A. Suppose that $C^{\prime }\widehat{A}C\stackrel{d}{\to }{C}^{\prime }\tilde{A}C$ for a random matrix, $\tilde{A}$, and for all $C\in \mathcal{H}$. Then, we have:

$$\begin{array}{c}\hfill \widehat{C}\stackrel{d}{\to }\tilde{C}\equiv arg\underset{C\in \mathcal{H}}{min}{C}^{\prime }\tilde{A}C.\end{array}$$

Thus, $\widehat{C}$ is random, even in the limit.

Let $\widehat{c}\left(S_i\right)$ and $\tilde{c}\left(S_i\right)$ be the i-th element of $\widehat{C}$ and $\tilde{C}$, respectively. Furthermore, let ${\widehat{\mu }}_{\mathrm{opt}}={\sum }_{i=1}^M\widehat{c}\left(S_i\right){\widehat{\mu }}_{S_i}$ denote the averaging estimator using $\widehat{C}$. Because $\widehat{c}\left(S_i\right)$ and ${\widehat{\mu }}_{S_i}$ are both determined through $D_n$, $\widehat{c}\left(S_i\right)$ and $\sqrt{n}({\widehat{\mu }}_{S_i}-{\mu }_{\mathrm{true}})$ converge jointly to $\tilde{c}\left(S_i\right)$ and ${\Lambda }_{S_i}$. Therefore, the limiting distribution of ${\widehat{\mu }}_{\mathrm{opt}}$ is given by:

$$\begin{array}{c}\hfill \sqrt{n}\left({\widehat{\mu }}_{\mathrm{opt}}-{\mu }_{\mathrm{true}}\right)\stackrel{d}{\to }\sum _{i=1}^M\tilde{c}\left(S_i\right){\Lambda }_{S_i}\stackrel{d}{=}{\Lambda }_0+w^{\prime }\left(\delta -\sum _{i=1}^M\tilde{c}\left(S_i\right)G_{S_i}D\right).\end{array}$$

(5)

Because weights are random, the limiting distribution is no longer normal. Thus, (5) is not readily applicable for inference. However, as suggested by Hjort and Claeskens [21], (5) implies that:

$$\begin{array}{c}\hfill \frac{1}{\widehat{\kappa }}\left[\sqrt{n}\left({\widehat{\mu }}_{\mathrm{opt}}-{\mu }_{\mathrm{true}}\right)-{\widehat{w}}^{\prime }\left(D_n-\sum _{i=1}^M\widehat{c}\left(S_i\right){\widehat{G}}_{S_i}{D}_n\right)\right]\stackrel{d}{\to }N(0,1),\end{array}$$

where $\widehat{\kappa }$ is a consistent estimator for ${({\tau }_0^2+w^{\prime }Qw)}^{1/2}$. This result can be used to construct a confidence interval for ${\mu }_{\mathrm{true}}$.

5. Example

This section gives a simple example to which our methods are applicable. One of the most popular models described by moment restrictions is the linear instrumental variable model. The full model we consider here is:

$$\begin{array}{ccc}\hfill y_i& =& x_i^{\prime }\theta +z_{1i}^{\prime }\gamma +u_i,\hfill \\ \hfill E\left[z_i{u}_i\right]& =& 0,\hfill \end{array}$$

where $x_i$ and $z_{1i}$ are $p\times 1$ and $q\times 1$ vectors of explanatory variables. Some elements of $x_i$ are potentially correlated with $u_i$. The vector of instruments, $z_i$, is $l\times 1$, which may contain elements of $x_i$ and $z_{1i}$. Economic theory suggests that $x_i$ should be included in the model, but we are unsure which components of $z_{1i}$ should be included. Thus, the reduced model corresponds to the case that $\gamma ={\gamma }_0={(0,\cdots ,0)}^{\prime }$.

In this model, $J_{\text{full}}$ is given by:

$$\begin{array}{c}\hfill J_{\mathrm{full}}=\left(\begin{array}{cc}{J}_{00}& J_{01}\\ J_{10}& J_{11}\end{array}\right)=\left(\begin{array}{cc}E\left[x_i{z}_i^{\prime }\right]E{\left[z_i{z}_i^{\prime }{u}_i^2\right]}^{-1}E\left[z_i{x}_i^{\prime }\right]& E\left[x_i^{\prime }{z}_i\right]E{\left[z_i{z}_i^{\prime }{u}_i^2\right]}^{-1}E\left[z_i{z}_{1i}^{\prime }\right]\\ E\left[z_{1i}{z}_i^{\prime }\right]E{\left[z_i{z}_i^{\prime }{u}_i^2\right]}^{-1}E\left[z_i{x}_i^{\prime }\right]& E\left[z_{1i}^{\prime }{z}_i\right]E{\left[z_i{z}_i^{\prime }{u}_i^2\right]}^{-1}E\left[z_i{z}_{1i}^{\prime }\right]\end{array}\right).\end{array}$$

Let ${\widehat{u}}_i$ be the residual from the full model: ${\widehat{u}}_i=y_i-x_i^{\prime }{\widehat{\theta }}_{\mathrm{full}}-z_{1i}^{\prime }{\widehat{\gamma }}_{\mathrm{full}}$. Then, for instance, $J_{00}$ can be estimated by:

$$\begin{array}{c}\hfill {\widehat{J}}_{00}=\frac{1}{n}\sum _{i=1}^n{x}_i{z}_i^{\prime }{\left(\frac{1}{n}\sum _{i=1}^n{z}_i{z}_i^{\prime }{\widehat{u}}_i^2\right)}^{-1}\frac{1}{n}\sum _{i=1}^n{z}_i{x}_i^{\prime }.\end{array}$$

(6)

Other components of $J_{\mathrm{full}}$ can be estimated in a similar manner. It also is possible to replace the empirical probability, $n^{-1}$, with the GEL-induced probability.

If the focus parameter is the k-th element of $\theta$, then we have:

$$\begin{array}{c}\hfill \widehat{w}={\widehat{J}}_{10}{\widehat{J}}_{00}^{-1}{e}_k,\end{array}$$

where $e_k$ is the k-th unit vector, which have one in the k-th element and zero, elsewhere. On the other hand, if the focus parameter is $\mu (\theta ,\gamma )=x^{\prime }\theta +z_1^{\prime }\gamma$ for a fixed covariate value ${(x^{\prime },z_1^{\prime })}^{\prime }$, then:

$$\begin{array}{c}\hfill \widehat{w}={\widehat{J}}_{10}{\widehat{J}}_{00}^{-1}x-z_1.\end{array}$$

To obtain a good estimate of $x^{\prime }\theta +z_1^{\prime }\gamma$ for a range of covariate values, rather than a single covariate value, we can utilize the idea of Claeskens and Hjort [36], who address minimizing an averaged risk over the range of covariates, rather than the pointwise risk.

6. Monte Carlo Study

We now investigate the performance of post-selection and averaging estimators by a simple Monte Carlo study. Our EL-based methods are compared with EL-based selection methods of Hong, Preston and Shum [16]. The following post-selection and averaging estimators are considered: (i) AIC-like model selection (ii) BIC-like model selection, (iii) FIC model selection and (iv) an averaging estimator, whose weights are given by (4). AIC- and BIC-like criteria are proposed by Hong, Preston and Shum [16] and are given by:

$$\begin{array}{ccc}\hfill {\mathrm{AIC}}_S& =& 2\sum _{i=1}^{n}log(1-{\widehat{\tau }}_S^{\prime }m(y_i,{\widehat{\theta }}_S,{\widehat{\gamma }}_S))-2(l-p-q_S),\hfill \\ \hfill {\mathrm{BIC}}_S& =& 2\sum _{i=1}^{n}log(1-{\widehat{\tau }}_S^{\prime }m(y_i,{\widehat{\theta }}_S,{\widehat{\gamma }}_S))-(l-p-q_S)logn.\hfill \end{array}$$

We use (6) to estimate J.

We consider the linear instrumental variable model. The DGP is specified by the following equations:

$$\begin{array}{ccc}\hfill y_i& =& {\theta }_0+{\theta }_1{x}_i+{\theta }_2{z}_{1i}+\sum _{k=1}^4{\gamma }_{kn}{z}_{k+1,i}+u_i,\hfill \\ \hfill x_i& =& 0.3z_{6i}+0.2z_{7i}+0.5u_i,\hfill \end{array}$$

where ${({\theta }_0,{\theta }_1,{\theta }_2)}^{\prime }={(1,1,1)}^{\prime }$ and ${({\gamma }_{1n},{\gamma }_{2n},{\gamma }_{3n},{\gamma }_{4n})}^{\prime }=\delta /\sqrt{n}$ for some vector $\delta ={({\delta }_1,{\delta }_2,{\delta }_3,{\delta }_4)}^{\prime }$. Exogenous variables, $z_{1i},\cdots ,z_{7i}$, are normally distributed with mean zero and variance one, and the correlation between $z_{ki}$ and $z_{li}$ is $0.5^{|k-l|}$ for $k\ne l$. The vector of instruments is fixed to be $z_i={(1,z_{1i},\cdots ,z_{7i})}^{\prime }$. The error term, $u_i$, is independent of $z_{1i},\cdots ,z_{7i}$ and is generated from a standard normal distribution. Thus, the moment restriction for the full model is:

$$\begin{array}{c}\hfill E_n\left[z_i\left(y_i-{\theta }_0-{\theta }_1{x}_i-{\theta }_2{z}_{1i}-\sum _{k=1}^4{\gamma }_{kn}{z}_{k+1,i}\right)\right]=0.\end{array}$$

Table 1. Estimation results; DGP, data generating process; AIC, Akaike information criterion; BIC, Bayesian information criterion; FIC, focused information criterion.

		DGP
		(1)	(2)	(3)	(4)
Full	Bias	-0.104	-0.109	- 0.089	- 0.076
	Std	0.544	0.533	0.509	0.489
	RMSE	0.554	0.544	0.516	0.495
Reduced	Bis	-0.279	-0.057	-0.148	-0.048
	Std	0.780	0.473	0.955	0.448
	RMSE	0.828	0.477	0.965	0.450
AIC	Bias	-0.113	-0.099	-0.101	-0.079
	Std	0.559	0.557	0.497	0.509
	RMSE	0.570	0.566	0.507	0.515
BIC	Bias	-0.136	-0.088	-0.104	-0.073
	Std	0.689	0.552	0.499	0.502
	RMSE	0.702	0.559	0.510	0.507
FIC	Bias	-0.139	-0.095	-0.112	-0.076
	Std	0.530	0.509	0.464	0.452
	RMSE	0.548	0.517	0.477	0.458
Averaging	Bias	-0.139	-0.092	-0.107	-0.074
	Std	0.511	0.476	0.455	0.444
	RMSE	0.529	0.484	0.468	0.450

Table 1. Estimation results; DGP, data generating process; AIC, Akaike information criterion; BIC, Bayesian information criterion; FIC, focused information criterion.

		DGP
		(1)	(2)	(3)	(4)
Full	Bias	-0.104	-0.109	- 0.089	- 0.076
	Std	0.544	0.533	0.509	0.489
	RMSE	0.554	0.544	0.516	0.495
Reduced	Bis	-0.279	-0.057	-0.148	-0.048
	Std	0.780	0.473	0.955	0.448
	RMSE	0.828	0.477	0.965	0.450
AIC	Bias	-0.113	-0.099	-0.101	-0.079
	Std	0.559	0.557	0.497	0.509
	RMSE	0.570	0.566	0.507	0.515
BIC	Bias	-0.136	-0.088	-0.104	-0.073
	Std	0.689	0.552	0.499	0.502
	RMSE	0.702	0.559	0.510	0.507
FIC	Bias	-0.139	-0.095	-0.112	-0.076
	Std	0.530	0.509	0.464	0.452
	RMSE	0.548	0.517	0.477	0.458
Averaging	Bias	-0.139	-0.092	-0.107	-0.074
	Std	0.511	0.476	0.455	0.444
	RMSE	0.529	0.484	0.468	0.450

The focus parameter is $\mu ={\theta }_1$. In many applications, it is often the case that the only parameter of interest in the linear model is the coefficient of the endogenous regressor. Exogenous regressors are included simply to avoid omitted variable bias. Thus, if the bias is small, it may be better to exclude some regressors to reduce the variance. In this simulation, we include the constant term, $x_i$, and $z_{1i}$ in all candidate models, but some elements of ${(z_{2i},z_{3i},z_{4i},z_{5i})}^{\prime }$ may be excluded. That is, some elements of ${({\gamma }_{1n},{\gamma }_{2n},{\gamma }_{3n},{\gamma }_{4n})}^{\prime }$ are set to zero. Therefore, there are $2^4=16$ submodels in total.

To evaluate the performance of the post-selection and averaging estimators, we calculate the bias, standard deviation and root MSE (RMSE) of each estimator over 1,000 repetitions. For reference, we also report the results of the full and reduced models. The sample size is $n=50$.2 We consider four DGPs: (1) $\delta ={(1,1,1,1)}^{\prime }$, (2) $\delta ={(\frac{1}{8},\frac{1}{8},\frac{1}{8},\frac{1}{8})}^{\prime }$, (3) $\delta ={(1,\frac{3}{4},\frac{1}{2},\frac{1}{4})}^{\prime }$ and (4) $\delta ={(\frac{1}{4},\frac{3}{16},\frac{1}{8},\frac{1}{16})}^{\prime }$. The DGPs (1) and (3) are favorable for the full model, while (2) and (4) are favorable for the reduced model. The results are summarized in Table 1.

Table 1 indicates that there are certain cases where we should avoid using the full model, even if it is the correct model. Performance of the full model is poorer than the FIC-selected model for all GDPs. As the theory suggests, the efficiency gain of FIC over the full model is large when $\delta$ is small. The averaging estimator outperforms all post-selection estimators. It is even better than FIC. As is consistent with findings in the literature, averaging is a useful method to reduce the risk of the estimator.

7. Conclusions

This paper studied GEL-based model selection and averaging methods that are designed to obtain an efficient estimator for the parameter of interest. We modified the local misspecification framework of Claeskens and Hjort [7], so that an FIC can be obtained for moment restriction models. Then, we proposed the averaging estimator by extending the idea of FIC.

In the simulation study, we considered the model selection/averaging problem for the linear instrumental variable model. Although some methods have been advocated for selecting/averaging instruments in the literature, there are few studies on the model selection/averaging problem. The result of the simulation suggests that our averaging can be a useful alternative to existing post-selection estimators.