On the Stock–Yogo Tables

Abstract

A standard test for weak instruments compares the first-stage F-statistic to a table of critical values obtained by Stock and Yogo (2005) using simulations. We derive a closed-form solution for the expectation from which these critical values are derived, as well as present some second-order asymptotic approximations that may be of value in the presence of multiple endogenous regressors. Inspection of this new result provides insights not available from simulation, and will allow software implementations to be generalised and improved. Finally, we explore the calculation of p-values for the first-stage F-statistic weak instruments test.

1. Introduction

In a seminal contribution, Phillips (1989) focussed the attention of the profession on the distributional consequences of what has come to be known as the problem of weak instruments. There followed a series of important papers that helped crystallise the consequences for inference of this problem including, but not restricted to, Nelson and Startz (1990a, 1990b); Buse (1992); Cragg and Donald (1993); Dufour (1997) and Kleibergen (2002). There have been many, many responses seeking to address the issues raised in this literature, some with greater merit than others. Phillips continues to make substantial contributions to this area of the literature. See, for example, Phillips (2016, 2017); Phillips and Gao (2017). One development that, for better or worse,1 has had significant practical impact, as a consequence of its inclusion in widely used econometric software, is that of Stock and Yogo (2005), hereafter SY. It is this latter development which is the focus of this paper.

The fundamental contribution of SY was to develop quantitative definitions of weak instruments, based on either IV estimator bias or Wald test size distortion, that were testable. Their idea was to relate the first-stage F-statistic (or, when there are multiple endogenous regressors, the Cragg–Donald (Cragg and Donald 1993) statistic) to a non-centrality parameter that, in turn, was related to the aforementioned estimator bias or test size distortion. In this way, they were able to use this F-statistic to test whether instruments were weak.2 That such tests have become part of the toolkit of many practitioners is evidenced by the fact that critical values for the SY tests are available within standard computer software, such as Stata (StataCorp 2015, e.g.,) when using either the intrinsic ivregress command or the ivreg2 package (Baum et al. 2010). The difficulty in the SY approach is that, in order to compute appropriate critical values, it is necessary to evaluate a complicated integral as an intermediate step. SY did this by Monte Carlo simulation and the tables of critical values they provided are widely used in practice.

In this paper, we focus primarily on the two-stage least squares (2SLS) bias representation of weak instruments for the single endogenous variable case, which we shall call the scalar case. Although the SY tables allow for more endogenous regressors, Sanderson and Windmejer (2016) demonstrate how these situations can all be mapped into the single endogenous regressor case, making it the case of greatest interest and importance. We show that, for this case, the integral mentioned above need not be estimated by simulation methods, as it can be resolved analytically and evaluated numerically using the intrinsic functions of software such as Matlab (MathWorks 2016). This result, Theorem 1, is presented in the next section, in which we also provide complete details of the model in question and the problem to be addressed.

From an empirical perspective, there are two important consequences of Theorem 1. First, it allows us to examine the accuracy of the SY critical values that have become so important in empirical research. For the most part, these critical values concord reasonably well with those that we derive analytically, although the most substantial differences occur in regions that we would argue are of practical significance.3 Second, it is now straightforward to generate more extensive sets of critical values, something that we do in

Table 1. 5% Critical values $\left(c{v}_{0.05}^{SW}\right)$ for single endogenous regressor, 2SLS bias.

${\mathit{k}}_{\mathit{z}}\setminus \mathit{B}$	0.01	0.05	0.1	0.15	0.2	0.25	0.3
2	11.57	9.02	7.85	7.14	6.61	6.19	5.83
3	46.32	13.76	9.18	7.52	6.60	5.96	5.49
4	63.10	16.72	10.23	7.91	6.67	5.88	5.32
5	72.55	18.27	10.78	8.11	6.71	5.82	5.19
6	78.59	19.19	11.08	8.21	6.70	5.75	5.09
7	82.75	19.79	11.25	8.25	6.67	5.69	5.01
8	85.78	20.20	11.36	8.26	6.64	5.63	4.93
9	88.07	20.49	11.42	8.25	6.60	5.58	4.87
10	89.86	20.70	11.46	8.24	6.56	5.52	4.81
11	91.30	20.86	11.49	8.22	6.53	5.48	4.76
12	92.47	20.99	11.50	8.20	6.49	5.43	4.71
13	93.43	21.08	11.50	8.17	6.46	5.39	4.67
14	94.25	21.16	11.50	8.15	6.42	5.36	4.63
15	94.94	21.22	11.49	8.13	6.39	5.32	4.59
16	95.54	21.26	11.49	8.11	6.36	5.29	4.56
17	96.05	21.30	11.48	8.08	6.34	5.26	4.53
18	96.50	21.33	11.46	8.06	6.31	5.23	4.50
19	96.09	21.35	11.45	8.04	6.29	5.21	4.47
20	97.25	21.37	11.44	8.02	6.26	5.18	4.45
21	97.56	21.39	11.43	8.00	6.24	5.16	4.43
22	97.84	21.40	11.41	7.98	6.22	5.14	4.40
23	98.09	21.41	11.40	7.96	6.20	5.12	4.38
24	98.32	21.41	11.39	7.94	6.18	5.10	4.36
25	98.53	21.42	11.38	7.93	6.16	5.08	4.35
26	98.71	21.42	11.36	7.91	6.15	5.06	4.33
27	98.88	21.42	11.35	7.90	6.13	5.05	4.31
28	99.04	21.42	11.34	7.88	6.11	5.03	4.30
29	99.18	21.42	11.32	7.87	6.10	5.02	4.28
30	99.31	21.42	11.31	7.85	6.08	5.00	4.27

(also in Section 2). In particular, we extend the SY tables to include more values of both $k_z$ and B, where $k_z$ is the number of instruments and B denotes the bias of 2SLS relative to that of ordinary least squares (OLS).

From a theoretical perspective, Theorem 1 provides a foundation that allows us to explore analytically certain patterns that exist in the SY tables, something that can only be alluded to on the basis of simulation results. These cases are explored in Section 3 of the paper. To further support the discussion of Section 3, we present in Section 4 some Monte Carlo simulation results, where we explore the sampling distributions of the F-statistics in relation to the bias of the 2SLS estimator relative to that of the OLS estimator, for the $k_z=2$ and $k_z=3$ cases.

Key to the development of Theorem 1 is the expectation of the ratio of a bilinear form in perfectly correlated normally distributed random variables, that differ only in their means, to a quadratic form in one of these same random variables, which is of some independent interest. We note in passing that the problem could be re-cast as one involving the expectation of a ratio of quadratic forms in normal variables, although in this form the normal variables have a singular distribution and both the numerator and the denominator weighting matrices are also singular, with the numerator weighting matrix asymmetric. This observation explains the difficulty in evaluating the integral analytically, but is also the reason that the expectation ultimately has such a simple structure. This expectation is evaluated in Appendix A.

Given the recent statement on p-values issued by the American Statistical Association Board of Directors (Wasserstein and Lazar 2016), it would be remiss of a paper such as this to be silent on the matter. In Section 5, we extend our discussion to show how p-values can be readily calculated on the basis of our earlier results.

Having extensively studied the scalar case, in Section 6, we turn our attention to analysing the model when there are multiple endogenous regressors in the model of interest. We shall, hereafter, refer to this as the general case. We find that we are able to draw on a variety of results developed in the exact finite sample literature. Not that any of our results are exact, they are all asymptotic in nature, but they do share a common structure with the earlier work that we are able to exploit. In addition to obtaining results for the relative bias of 2SLS, in the general case, we also provide first- and second-order asymptotic approximations to the relative bias, where the nesting sequence is the number of instruments. Overall, the analysis of the general case is less favourable to the SY approach than is the scalar case.

Final remarks appear in Section 8. For the most part, we have relegated technical developments to the various appendices, as well as discussion of some matters deemed secondary to the main ideas of the paper.

2. An Analytic Development of Stock–Yogo

Consider the simple model

$$y=x\beta +u,$$

(1)

where $y={[y_1,\dots ,y_n]}^{\prime }$, $x={[x_1,\dots ,x_n]}^{\prime }$ and $u={[u_1,\dots ,u_n]}^{\prime }$ are $n\times 1$ vectors, with n the number of observations. The regressor x is assumed endogenous, so that $E\left[u|x\right]\ne 0$. Other exogenous regressors in the model, including the constant, have been partialled out.

We can implicitly define a set of instruments via the following linear projection

$$x=Z\pi +v,$$

(2)

where Z is an $n\times k_z$ matrix of instruments (with full column rank), $\pi$ a $k_z\times 1$ vector of parameters and v is an $n\times 1$ error vector. In this model, $k_z-1$ is the degree of over-identification. We assume that individual observations are independently and identically distributed, and

$$\left[\begin{array}{c}{u}_i\\ v_i\end{array}\right]\mid z_i\sim \left(0,\mathsf{\Sigma }\right),\mathrm{with}\mathsf{\Sigma }=\left[\begin{array}{cc}{\sigma }_u^2& {\sigma }_{uv}\\ {\sigma }_{uv}& {\sigma }_v^2\end{array}\right],i=1,2,\dots ,n,$$

where $z_i^{\prime }$ denotes the ith row of Z. A test for $H_0:\pi =0$ against $H_1:\pi \ne 0$, is the so-called first-stage F-statistic

$$F=\frac{{\widehat{\pi }}^{\prime }{Z}^{\prime }Z\widehat{\pi }}{k_z{\widehat{\sigma }}_v^2}\underset{d}{\overset{H_0}{⟶}}\frac{{\chi }_{k_z}^2}{k_z},$$

(3)

where $\widehat{\pi }={\left(Z^{\prime }Z\right)}^{-1}{Z}^{\prime }x$ and ${\widehat{\sigma }}_v^2=n^{-1}{x}^{\prime }(I_n-Z{\left(Z^{\prime }Z\right)}^{-1}{Z}^{\prime })x$. Here, a large value of the statistic is evidence against the null hypothesis, which is that the nominated instruments are irrelevant.

Following Staiger and Stock (1997), we consider values of $\pi$ local to zero, as $\pi =c/\sqrt{n}$. We then obtain for the concentration parameter ${\mu }_n^2$,

$${\mu }_n^2=\frac{{\pi }^{\prime }{Z}^{\prime }Z\pi }{{\sigma }_v^2}\underset{p}{⟶}\frac{c^{\prime }{Q}_{zz}c}{{\sigma }_v^2}\equiv {\mu }^2,$$

where $Q_{zz}=E\left[z_i{z}_i^{\prime }\right]={\mathrm{plim}}_{n\to \infty }{n}^{-1}{Z}^{\prime }Z$ is positive definite by assumption. We see that $k_{z}F$ is a sample analogue of ${\mu }^2$. With this formulation, the testing problem previously discussed is equivalent to that of testing $H_0^{\prime }:{\mu }^2=0$ against $H_1^{\prime }:{\mu }^2>0$. Rather than testing for the irrelevance of instruments, SY characterised weak instruments as a situation where ${\mu }^2$ was greater than zero but proximate to it. Specifically, their testing problem can be thought of as $H_0^{″}:{\mu }^2={\mu }_0^2>0$ against $H_1^{″}:{\mu }^2>{\mu }_0^2$, for some suitably specified value of ${\mu }_0^2$. The statistic F is still a natural one in this problem; although, of course, the null distribution is no longer the central distribution associated with ${\mu }_0^2=0$. Instead, we have

$$F\underset{d}{\overset{H_0}{⟶}}\frac{{\chi }_{k_z,{\mu }_0^2}^{\prime 2}}{k_z},$$

(4)

where ${\chi }_{k,\delta }^{\prime 2}$ denotes a random variable following a non-central chi-squared distribution with k degrees of freedom and non-centrality parameter $\delta \ge 0$.4 Let

$${\chi }_{\alpha }={\chi }_{k_z,{\mu }_0^2}^{\prime 2}(1-\alpha )$$

denote the $(1-\alpha )100$th quantile of a non-central chi-squared distribution with $k_z$ degrees of freedom and non-centrality parameter ${\mu }_0^2$. Then, the relevant size $\alpha$ critical region is

$$\left\{F:F>c{v}_{\alpha }=\frac{{\chi }_{\alpha }}{k_z}\right\},$$

(5)

where ${\chi }_{\alpha }$ can be obtained, for given ${\mu }_0^2$ and $k_z$, as the solution to either of the equations

$$\begin{array}{cc}\hfill 1-\alpha & =e^{-{\mu }^2/2}\sum _{j=0}^{\infty }\frac{{({\mu }^2/2)}^j}{2^{k_z/2+j}j!\mathsf{\Gamma }\left(\frac{k_z}{2}+j\right)}{\int }_0^{{\chi }_{\alpha }}{e}^{-s/2}{s}^{k_z/2+j-1}\mathrm{d}s\hfill \end{array}$$

(6a)

$$\begin{array}{c}=e^{-{\mu }^2/2}{e}^{-{\chi }_{\alpha }/2}{\left(\frac{{\chi }_{\alpha }}{2}\right)}^{k_z/2}\sum _{j=0}^{\infty }\frac{{({\chi }_{\alpha }{\mu }^2/4)}^j}{j!\mathsf{\Gamma }\left(\frac{k_z+2}{2}+j\right)}{}_1{F}_1^{}\left(1;\frac{k_z+2}{2}+j;\frac{{\chi }_{\alpha }}{2}\right),\hfill \end{array}$$

(6b)

and where $1F_1^{}(·;·;·)$ denotes a confluent hypergeometric function (Slater 1960).

The aspect of the SY approach that remains outstanding is the choice of ${\mu }_0^2$. Their quantitative definition of the weakness of a set of instruments is couched in terms of the impact that it has on inference. They provided two possible definitions that variously reflect the known consequences of weak instruments for (i) estimation, through the bias of the estimator, and (ii) hypothesis testing, through the size of a particular Wald test relative to its nominal size. It is the former that is in most common use and the approach of interest here.5

In particular, SY relate the bias of the 2SLS estimator of $\beta$, ${\widehat{\beta }}_{2SLS}$, relative to that of the ordinary least squares estimator of $\beta$, ${\widehat{\beta }}_{OLS}$, to the first-stage F-statistic by showing that they are both related to ${\mu }^2$. A value for ${\mu }^2$, denoted ${\mu }_0^2$, is then chosen to allow a certain level of relative bias. Specifically, let $B_n$ denote the relative bias for a given sample size n, i.e.,

$$\left|B_n\right|=\left|\frac{E\left[{\widehat{\beta }}_{2SLS,n}\right]-\beta }{E\left[{\widehat{\beta }}_{OLS,n}\right]-\beta }\right|.$$

As discussed in Chao and Swanson (2007), if there exists a positive integer $N<\infty$ such that6

$$\begin{array}{c}\underset{n\ge N}{sup}E\left[{\left|{\widehat{\beta }}_{2SLS,n}-\beta \right|}^{1+\delta }\right]<\infty \end{array}$$

(7)

for some $\delta$ > 0, then the limit of the sequence of finite sample biases will coincide with the bias computed from the local-to-zero asymptotic distribution. That is,

$$\begin{array}{c}\underset{n\to \infty }{lim}|B_n|=\left|E\left[\frac{{(\xi -{\lambda }_0)}^{\prime }\xi }{{\xi }^{\prime }\xi }\right]\right|\equiv \left|B\right|,\end{array}$$

(8)

where $\xi \sim \mathrm{N}\left({\lambda }_0,I_{k_z}\right)$. The test for weak instruments then proceeds as follows:

• The practitioner chooses a value for $\left|B\right|$, e.g., $\left|B\right|=0.1$, if an asymptotic relative bias of less than 10% is deemed acceptable.
• Given $k_z$ and $\left|B\right|$, ${\mu }_0^2={\lambda }_0^{\prime }{\lambda }_0$ is obtained on solving (8).
• Given ${\mu }_0^2$, critical values for F can be determined, which are proportional to those of the non-central chi-squared distribution as specified in (4).
• The null of weak instruments is then rejected for sufficiently large values of the first-stage F-statistic, and we conclude that $\left|B\right|$ is no larger than the value chosen in Step 1 above.

The difficulty in the procedure just described is that, at Step 2, there is an integral that must be evaluated as part of a search for ${\mu }_0^2$. SY do this using a 20,000 draw Monte Carlo simulation. This is unnecessary as the integral can be solved analytically. The result is summarised in the following theorem.

Theorem 1.

If B is as defined in Equation (8), then, provided $k_z\ge 2$,

$$B={}_1{F}_1^{}\left(1;\frac{k_z}{2};-\frac{{\mu }_0^2}{2}\right)>0,$$

(9)

where, as noted following (6), ${}_1{F}_1(·;·;·)$ denotes a confluent hypergeometric function.

Proof. 

The result follows immediately from Theorem A1, in Appendix A, which establishes the equality, and from the observation that if $b\ge a>0$ but $s<0$ then $0<{}_1{F}_1^{}\left(a;b;s\right)\le 1$, which establishes the inequality.7 □

That Theorem 1 involves a confluent hypergeometric function is not surprising as they have long figured prominently in the finite sample literature; see, for example, Phillips (1980) and the papers cited therein. These functions have been very intensively studied in the mathematics literature over a period of hundreds of years and so an important consequence of Theorem 1 is that it allows the use of efficiently programmed intrinsic functions in readily available software, such as Matlab (MathWorks 2016), at each step of a search for ${\mu }_0^2$ rather than having to estimate an integral by simulation.8 For the special case of $k_z=2$,

$$1F_1^{}\left(1;\frac{k_z}{2};-\frac{{\mu }_0^2}{2}\right)=exp\left\{-\frac{{\mu }_0^2}{2}\right\},$$

(10)

making evaluation of the expression especially simple.

Using our result, we provide in Table 1 an extended version of that panel of SY Table 5.1 corresponding to a single endogenous variable, which is the set of critical values most commonly used. We note that SY start their tables at $k_z=3$ even though, following the arguments of Kinal (1980), finite biases will exist for all $k_z\ge 2$ if one is prepared to make a normality assumption. As this is a practically relevant case, we include it in Table 1. However, such inclusion is not without controversy. The mode of convergence leading to Label (8) is convergence in distribution. Existence of moments is not sufficient to imply convergence in expectation, which is a stronger result (see Label (7) and the accompanying discussion). Heuristically, (7) might be interpreted as meaning that a little more than simply the existence of the moments of the estimators is required for the sequence of biases to converge to the local-to-zero asymptotic results, and so this might be achieved by requiring $k_z\ge 3$ in the case of a single endogenous regressor rather than just $k_z\ge 2$, although we note that (7) doesn’t actually say this. In any event, the inclusion in Table 1 of the row $k_z=2$ may be viewed as something of an ad hoc approximation. Some confidence in the value of the approximation may be garnered from the simulation results presented in Section 4.

Where Table 1 overlaps with SY (Table 5.1), we are able to provide an indication of the difference made by the analytical evaluation of the expectation in (8). As shown in

Table 2. Differences: $c{v}_{0.05}^{SW}-c{v}_{0.05}^{SY}$.

${\mathit{k}}_{\mathit{z}}\setminus \mathit{B}$	0.05	0.10	0.20	0.30
3	0.15	−0.10	−0.14	−0.10
4	0.13	0.04	0.04	0.02
5	0.10	0.05	0.06	0.06
6	0.09	0.04	0.06	0.06
7	0.07	0.04	0.06	0.07
8	0.05	0.03	0.05	0.06
9	0.04	0.04	0.05	0.05
10	0.04	0.03	0.05	0.05
11	0.04	0.01	0.03	0.04
12	0.02	0.02	0.04	0.04
13	0.02	0.02	0.03	0.04
14	0.02	0.02	0.03	0.04
15	0.01	0.02	0.03	0.04
16	0.02	0.01	0.03	0.03
17	0.01	0.01	0.02	0.03
18	0.01	0.02	0.02	0.03
19	0.01	0.01	0.02	0.04
20	0.01	0.01	0.02	0.03
21	0.00	0.01	0.02	0.03
22	0.00	0.01	0.02	0.03
23	0.00	0.01	0.02	0.03
24	0.00	0.01	0.02	0.03
25	0.00	0.00	0.02	0.02
26	0.00	0.01	0.01	0.02
27	0.00	0.01	0.01	0.03
28	0.00	0.02	0.02	0.02
29	0.00	0.01	0.01	0.03
30	0.00	0.01	0.01	0.02

Note: The values of $c{v}_{0.05}^{SY}$ are taken from Stock and Yogo (2005), Table 5.1.

, the differences are typically small, with the largest differences when $k_z$ and B are themselves small, which we would argue is the most important case in practice.9

3. Some Further Consequences of Theorem 1

Theorem 1 allows us to prove a variety of further results that can only be speculated about on the basis of simulation results.

Remark 1.

Implicit in Theorem 1 is the observation that, whenever $k_z\ge 2$, OLS and 2SLS are always asymptotically biased in the same direction, making the absolute value function of $\left|B\right|$ in (8) redundant.

Remark 2.

The values of the limiting relative biases of ${\widehat{\beta }}_{2SLS}$ and ${\widehat{\beta }}_{OLS}$ are explored in Figure 1 for different values of the parameters $k_z$ and ${\mu }^2/2$. The figure illustrates that, for $k_z\ge 2$, the function is increasing in its argument, which is $-{\mu }^2/2$. Note also that, as ${\mu }^2\to 0$, the information in the instruments approaches zero, and so the local-to-zero asymptotic bias of ${\widehat{\beta }}_{2SLS}$ approaches that of ${\widehat{\beta }}_{OLS}$ from below. Hence, the limit of the relative asymptotic biases at ${\mu }^2=0$ is unity, which is the value of ${}_1{F}_1\left(1;\frac{k_z}{2};0\right)$.

Remark 3.

Certain patterns in Table 1 are readily established, as illustrated by the following result.10

Theorem 2.

The critical values $c{v}_{\alpha }$ are decreasing functions of B for given $k_z$.

Proof. 

See Appendix B.2. □

Heuristically, Theorem 2 states that the critical values will necessarily decrease as one moves from left to right across any given row of Table 1; that is, the critical values decrease as the practitioner is willing to accept increasing amounts of 2SLS bias relative to that of OLS. The intuition behind the results is as follows. An increase in B for fixed $k_z$ implies that the argument of the confluent hypergeometric function in (9) must increase, i.e., that ${\mu }^2/2$ must decrease. As ${\mu }^2$ approaches zero, the non-central chi-squared distribution from which critical values are drawn approaches a central chi-squared and the corresponding quantiles become smaller. Hence, as one moves across columns from left to right in Table 1, the $c{v}_{\alpha }$ become smaller.

Theorem 2 explains the row behaviour of Table 1. Explaining the column behaviour is much more complicated. Observation suggests the following to be true.

Conjecture 1.

For given B, the critical values $c{v}_{\alpha }$, presented in Table 1, are increasing functions of $k_z$ up to some value, k say, whereafter they are decreasing functions of $k_z$. k is a decreasing function of B.

Some intuition for Conjecture 1 is available from the definition of $c{v}_{\alpha }$, see (5), if one considers the impact of increasing the number of instruments by one, from $k_z$ to $k_z+1$, with superscripts ‘0’ and ‘1’ distinguishing the two cases, respectively. For given B and $\alpha$,

$$c{v}_{\alpha }^1-c{v}_{\alpha }^0⪌0\mathrm{as}\frac{{\chi }_{\alpha }^1-{\chi }_{\alpha }^0}{{\chi }_{\alpha }^0}⪌\frac{1}{k_z}.$$

k is then that value of $k_z$ after which the $c{v}_{\alpha }$ start diminishing.

Remark 4.

Although B does not exist when $k_z=1$, the confluent hypergeometric function of Theorem 1 remains well-defined. In Appendix D, we analyse the properties of ${}_1{F}_1\left(1;\frac{1}{2};-\frac{{\mu }^2}{2}\right)$.

4. Some Monte Carlo Results

We follow Sanderson and Windmejer (2016) and specify the model is as in (1) and (2), with $\beta =1$ and

$$\left[\begin{array}{c}{u}_i\\ v_i\end{array}\right]\sim \mathrm{N}\left(\left[\begin{array}{c}0\\ 0\end{array}\right],\left[\begin{array}{cc}1& 0.5\\ 0.5& 1\end{array}\right]\right).$$

The instruments in Z are $k_z$ independent standard normally distributed random variables and $\pi =\left(c_{B{k}_z}{\iota }_{k_z}\right)/\sqrt{n}$, where ${\iota }_{k_z}$ is a $k_z$ vector of ones, and with $c_{B{k}_z}$ chosen such that the relative bias B is equal to $0.01$, $0.05$, $0.10$ or $0.20$, for values of $k_z=3$ and $k_z=2$. The sample size $n=$ 10,000 and the results are presented in

Table 3. Simulation results for $k_z=3$ and $k_z=2$.

	B
	0.01		0.05		0.10		0.20
$k_z=3$	mean	std dev	mean	std dev	mean	std dev	mean	std dev
${\widehat{\beta }}_{OLS}$	1.4950	0.0086	1.4989	0.0087	1.4994	0.0087	1.4997	0.0087
${\widehat{\beta }}_{2SLS}$	1.0054	0.0998	1.0241	0.2222	1.0506	0.3161	1.1025	0.4276
F	34.713	6.7626	8.0336	3.1828	4.7849	2.3952	3.0948	1.8630
rel bias	0.0108		0.0482		0.1014		0.2052
${\mu }_0^2/k_z$	33.674		7.0445		3.7754		2.0902
cv F	46.316		13.765		9.1815		6.5960
rej freq F	0.0515		0.0505		0.0508		0.0511
$k_z=2$	mean	std dev	mean	std dev	mean	std dev	mean	std dev
${\widehat{\beta }}_{OLS}$	1.4996	0.0087	1.4997	0.0087	1.4997	0.0087	1.4998	0.0087
${\widehat{\beta }}_{2SLS}$	1.0056	0.4398	1.0256	0.7195	1.0519	0.9651	1.0981	1.1404
F	5.6124	3.1989	4.0004	2.6492	3.2963	2.3746	2.6011	2.0502
rel bias	0.0111		0.0513		0.1039		0.1962
${\mu }_0^2/k_z$	4.6052		2.9957		2.3026		1.6094
cv F	11.572		9.0232		7.8521		6.6087
rej freq F	0.0509		0.0507		0.0505		0.0498

Notes: Sample size n = 10,000, number of Monte Carlo replications is 100,000.

for 100,000 Monte Carlo replications.

For $k_z=3$, the results are exactly in line with the theory: the Monte Carlo relative biases are equal to B and the rejection frequencies of the first-stage F-test are 5% at the 5% nominal level, using the critical values reported in Table 1.

The results for $k_z=2$ are also in line with the theory, although we see here that the standard deviations of ${\widehat{\beta }}_{2SLS}$ are much larger than those of the $k_z=3$ case at the same values of B. This is due to the fact that the information needed to obtain the same relative bias is much smaller for the $k_z=2$ case than for the $k_z=3$ case, as reflected by their smaller ${\mu }_0^2/k_z$ values, but it also reflects the problem that the second moment does not exist when the degree of over-identification is equal to 1. The interquartile ranges for the 2SLS estimator when $k_z=2$ are $0.3296,0.4170,0.4811$ and $0.5570$ for $B=0.01,0.05,0.10$ and $0.20$, respectively. These Monte Carlo results therefore confirm our theoretical findings for the $k_z=2$ case. Clearly some caution should be exercised when working with 2SLS in this case because it possesses no second moment.

5. $\mathit{p}$-Values

p-values are readily available as a straightforward extension of our earlier analysis. Specifically, from (4), we have the limiting result

$$k_z\times F\underset{d}{\overset{H_0}{⟶}}{\chi }_{k_z,{\mu }_0^2}^{\prime 2}.$$

(11)

For any particular sample value of the F-test, say $\widehat{F}$, if $X\sim {\chi }_{k_z,{\mu }_0^2}^{\prime 2},$ then the p-value for the SY weak instruments test considered in this paper is simply $\mathrm{Pr}\left(X\ge k_z\times \widehat{F}\right)$. Of course, the problem here is the determination of ${\mu }_0^2$.

Table 4. Values for ${\mu }_0^2/k_z$ corresponding to Table 1.

${\mathit{k}}_{\mathit{z}}\setminus \mathit{B}$	0.01	0.05	0.1	0.15	0.2	0.25	0.3
02	04.605	02.996	2.303	1.897	1.609	1.386	1.204
03	33.674	07.045	3.775	2.677	2.090	1.706	1.426
04	50.000	10.000	5.000	3.329	2.483	1.960	1.599
05	59.799	11.793	5.784	3.774	2.761	2.144	1.724
06	66.332	12.991	6.315	4.081	2.958	2.277	1.816
07	70.998	13.848	6.696	4.304	3.102	2.375	1.885
08	74.498	14.491	6.982	4.472	3.212	2.450	1.938
09	77.221	14.992	7.205	4.604	3.298	2.510	1.980
10	79.398	15.392	7.384	4.709	3.367	2.558	2.014
11	81.180	15.720	7.531	4.796	3.424	2.597	2.043
12	82.665	15.993	7.653	4.868	3.471	2.630	2.066
13	83.922	16.224	7.756	4.929	3.511	2.658	2.086
14	84.999	16.423	7.845	4.981	3.546	2.682	2.104
15	85.932	16.594	7.922	5.027	3.576	2.703	2.119
16	86.749	16.745	7.989	5.067	3.602	2.721	2.132
17	87.470	16.877	8.048	5.102	3.626	2.738	2.144
18	88.110	16.995	8.101	5.133	3.646	2.752	2.154
19	88.683	17.101	8.148	5.161	3.665	2.765	2.163
20	89.199	17.196	8.191	5.186	3.681	2.777	2.172
21	89.666	17.281	8.229	5.209	3.697	2.787	2.179
22	90.090	17.360	8.264	5.230	3.710	2.797	2.186
23	90.477	17.431	8.296	5.249	3.723	2.806	2.193
24	90.833	17.496	8.326	5.266	3.734	2.814	2.198
25	91.159	17.556	8.353	5.282	3.745	2.821	2.204
26	91.461	17.612	8.377	5.297	3.755	2.828	2.209
27	91.740	17.663	8.400	5.311	3.764	2.834	2.213
28	91.999	17.711	8.422	5.323	3.772	2.840	2.217
29	92.241	17.755	8.442	5.335	3.780	2.846	2.221
30	92.466	17.797	8.460	5.346	3.787	2.851	2.225

reports those values of ${\mu }_0^2/k_z$ that were calculated in order to construct Table 1. For those values of B considered in Table 1, we now have the parameters $k_z$ and ${\mu }_0^2/k_z$. Consequently, any computer software that can evaluate a non-central chi-squared cdf can readily calculate p-values for the test for weak instruments considered here.

6. Multiple Endogenous Regressors

The general case is obtained by defining x in (1) to be a matrix of endogenous regressors of dimension $n\times k_x$, say, so that $\beta$ is $k_x\times 1$. Then, (2) becomes a multivariate regression model with $\pi$ of dimension $k_z\times k_x$ and v of dimension $n\times k_x$. The rows of $[u,v]$ are independent with common covariance matrix $\mathsf{\Sigma }$. All other aspects of the model described in (1) and (2) remain essentially unchanged.11

In terms of the SY analysis, it clearly makes no sense to proceed in terms of the ratio

$$\frac{E\left[{\widehat{\beta }}_{2SLS}\right]-\beta }{E\left[{\widehat{\beta }}_{OLS}\right]-\beta },$$

given that $\beta$ is now a vector rather than a scalar quantity. Thus, instead, they focus attention on the quadratic form (SY, Equation 3.8)

$$B^2=\frac{{\rho }^{\prime }{h}^{\prime }h\rho }{{\rho }^{\prime }\rho }={\gamma }^{\prime }{h}^{\prime }h\gamma ,$$

where h is a matrix analogue of the expectation explored in Theorem 1 and $\gamma =\rho {\left({\rho }^{\prime }\rho \right)}^{-1/2}$, with $\rho$ defined in SY (p. 85). The essential feature of $\gamma$ is that it is not a function of $\lambda$. Despite being somewhat more tractable, the quadratic form tells us no more about the non-centrality parameter determining the bias of the 2SLS estimator than does

$$h\gamma =E\left[{\left({\xi }^{\prime }\xi \right)}^{-1}{\xi }^{\prime }(\xi -\lambda )\right]\gamma =\gamma -\mathcal{J},$$

(12)

say, where the elements of the $k_z\times k_x$ matrix $\xi$ are jointly distributed according to

$$vec\left(\xi \right)\sim \mathrm{N}(vec\left(\lambda \right),I_{k_z\times k_x}),$$

(13)

with $\lambda$ as defined in SY (p. 85).12 Again, we should stress that the normality of $\xi$ is a consequence of the local-to-zero asymptotic analysis and is not a strong distributional assumption. As $\gamma$ is independent of $\lambda$, it is sufficient to focus attention on the vector of expectations $\mathcal{J}$. If we let $e_i^{\prime }$ denote the ith row of the identity matrix $I_{k_x},$ then we have immediately13

$$\mathcal{J}=E\left[{\left({\xi }^{\prime }\xi \right)}^{-1}{\xi }^{\prime }\lambda \gamma \right]=\mathrm{etr}\left\{\mathsf{\Lambda }\right\}\sum _{j=0}^{\infty }\sum _{\varphi }\sum _{\psi \in \varphi ·\left[1\right]}\frac{{\left(\frac{k_z}{2}\right)}_{\varphi }}{j!{\left(\frac{k_z}{2}\right)}_{\psi }}{\theta }_{\psi }^{\varphi ,1}C(\mathsf{\Lambda },\gamma ,k_z),$$

(14)

where the ith element of the $k_x\times 1$ vector $C(\lambda ,\gamma ,k_z)$ is $C_{\psi }^{\varphi ,1}\left(k_z\mathsf{\Lambda },k_z\mathsf{\Lambda }\gamma e_i^{\prime }\right)$, with $\mathsf{\Lambda }={\lambda }^{\prime }\lambda /\left(2k_z\right)$.14 Here, $\varphi$ denotes ordered partitions of j into no more than $k_x$ parts, so that $\varphi =({\varphi }_1,{\varphi }_2,\dots ,{\varphi }_p)$ where the integers ${\varphi }_1,{\varphi }_2,\dots ,{\varphi }_p$ satisfy the restrictions (i) ${\varphi }_1\ge {\varphi }_2\ge \cdots \ge {\varphi }_p$, (ii) ${\sum }_{i=1}^p{\varphi }_i=j$, and (iii) $p\le k_x$. The symbol ${\sum }_{\varphi }$ then denotes the sum over all such partitions of j. For example, the ordered partitions of 2 are the so-called top partition $\left[2\right]$ and $(1,1)$.15 The invariant polynomials $C_{\psi }^{\varphi ,1}(·,·)$ and the symbol ${\sum }_{\psi \in \varphi ·\left[1\right]}$ are defined in Davis (1979), p. 465, with ${\theta }_{\psi }^{\varphi ,1}=C_{\psi }^{\varphi ,1}(I_{k_z},I_{k_z})/C_{\psi }\left(I_{k_z}\right)$ a constant that may be zero. These were developed as extensions (to two matrix arguments) of the zonal polynomials $C_{\psi }(·)$ originally due to James (1961).16 Finally, the generalised hypergeometric coefficients are defined as (Constantine 1963 Equation (26))

$${\left(a\right)}_{\kappa }=\prod _{i=1}^m{\left(a-\frac{1}{2}(i-1)\right)}_{k_i},\kappa =(k_1,\dots ,k_m),$$

where ${\left(b\right)}_n=b(b+1)\cdots (b+n-1)$, ${\left(b\right)}_0=1$ is the usual Pochhammer symbol or forward factorial (see Slater 1966, Appendix I).

Expressions like (14) are computationally problematic. The available evidence suggests that the series are typically slow to converge (Phillips 1983a, 1983b). Unfortunately, the invariant polynomials of Davis are tabulated only to low order and, to date, no algorithms have been derived for their computation.17 Consequently, until such time as the computational restrictions are lifted, the practical relevance of the result is limited and is offered here only for completeness. Better progress might be made working with one of the various approximation techniques that are available, e.g., the Laplace approximation used by Phillips (1983b) to extract approximate marginal distributions for IV estimators in this more general setting. In this case, however, we can further adapt the results of Hillier et al. (1984) to obtain many instrument approximations to (14). Specifically, analogous to their Equations (32) and (33), we have

$${\mathcal{J}}_i=e_i^{\prime }\gamma -(k_z-k_x-1)e_i^{\prime }{\left({\lambda }^{\prime }\lambda \right)}^{-1}\gamma +O\left(k_z^{-2}\right),$$

(15)

and

$$\begin{array}{cc}\hfill {\mathcal{J}}_i=e_i^{\prime }\gamma & -(k_z-k_x-1)[1+tr\left\{{\left({\lambda }^{\prime }\lambda \right)}^{-1}\right\}]e_i^{\prime }{\left({\lambda }^{\prime }\lambda \right)}^{-1}\gamma \hfill \\ & +(k_z-k_x-1)(k_z-k_x-2)e_i^{\prime }{\left({\lambda }^{\prime }\lambda \right)}^{-2}\gamma +O\left(k_z^{-3}\right),\hfill \end{array}$$

(16)

where ${\mathcal{J}}_i$ denotes the ith element of $\mathcal{J}$.18

Although these approximations are operational, in the presence of multiple endogenous regressors, we question whether the approach of SY is as sound as it is in the single endogenous regressor case. Our concern is rooted in the structure of the Davis polynomials themselves. For matrix arguments, X and Y with given indices k and l, respectively, the polynomials are ‘linear combinations of the distinct products of traces

$${(tr{X}^{a_1}{Y}^{b_1}{X}^{c_1}\cdots )}^{r_1}{(tr{X}^{a_2}{Y}^{b_2}{X}^{c_2}\cdots )}^{r_2}\cdots$$

of total degree k, l in the elements of X, Y, respectively (Davis 1979, p. 468). It is immediately apparent that local-to-zero asymptotic expression for the bias of 2SLS is not a function of the eigenvalues of the concentration parameter or, at least, not a function of them alone. Stock and Yogo (2005), p. 90, remark on this themselves when discussing certain numerical results, where they observe that 2SLS bias is decreasing in all eigenvalues of the concentration parameter for all values of (what we call) $k_z$. To focus on the smallest eigenvalue, as the Cragg–Donald statistic does, is, consequently, problematic. Another way of thinking of this problem is to consider the problem of determining the magnitude of a matrix and to ask which of the following three matrices is either largest or smallest:

$$diag\left(1,2,3\right),diag\left(2,2,2\right),diag\left(0,0,6\right).$$

While consideration of the smallest eigenvalue will lead to a particular choice, it is not clear that that choice actually has that much to do with the exact behaviour of the IV estimator, except in the scalar case. Indeed, for this reason, the SY results for multiple endogenous variable cases are only approximate and provide upper bounds on critical values for the Cragg–Donald minimum eigenvalue test. That the SY approach works in the case of a single endogenous regressor is a consequence of the commutative law of multiplication that allows us to extract scalars from products in ways that we can’t in more general matrix situations. In summary, the SY approach results in a well-posed problem in the case of a single endogenous regressor but a somewhat poorly-posed problem in the case of multiple endogenous regressors.

If one must deal with multiple endogenous regressors, then we prefer the approach of Sanderson and Windmejer (2016), who define weak identification as the rank of $\mathsf{\Pi }$ being local to a rank reduction of one, which is essentially a scalar problem. By reducing the problem in this way, the approach is reduced to a well-posed problem for which the single endogenous variable results apply, only the degrees of freedom need to be adjusted for the number of endogenous variables (see Sanderson and Windmejer (2016) for details).

7. The Wisdom of Hindsight: Some Historical Remarks

As discussed in Footnote 7, helpful referees drew our attention to the fact that there are other ways that we might have approached this problem than the one that we initially chose. Indeed, once one recognises the the structure of the problem, many results become available. However, before discussing some of these, we will again stress that the results derived in the previous sections are all asymptotic in nature, there are no underlying exact distributional assumptions beyond those originally made in Staiger and Stock (1997). What these results share with the exact distribution literature is integrals with similar structures and a common approach to resolving them. Furthermore, the parameterisations adopted here are different from the canonical forms underlying the exact distributional results and so the resultant expressions are different even if their structures are reminiscent of earlier results. A prime example of this is given by the similarities between the many, local-to zero, instrument approximations of Equations (15) and (16) and the large-sample approximations of (Hillier et al. (1984, Equations (32) and (33)).

It was noted in Footnote 7 that Chao and Swanson (2007) had derived local-to-zero asymptotic expressions for the bias of both 2SLS and OLS in the scalar case, so that they might bias correct the estimators, and that these provide an alternate path to Theorem 1. If moments were the focus of our attention, then we should note that the results of Chao and Swanson (2007) have the same structure as do those of Richardson (1968) and references cited therein, who first derived moments in the scalar case with arbitrary numbers of instruments and proved Basmann’s conjecture (Richardson 1968, Section 4.3) for the existence of moments. In the general case, we should, of course, be looking to Hillier et al. (1984) for results with the same structure as presented here and to Kinal (1980), who established existence criteria in the general case. Noting that the distributions of interest in the exact finite sample literature are different from the distributions thrown up by the local-to zero asymptotics, with different parameters, it might be argued that the exact results for misspecified models are closer in spirit to what we have here. In this event, we might argue that results for moments are implicit (but unrecognisable) in Hale et al. (1980) or, in a far more recognisable form in Knight (1982), for the scalar case, or Skeels (1995) in the general case. However, here moments are only of interest as an intermediate result to obtain the relative bias that is used to obtain a non-centrality parameter that determines the non-central chi-squared distribution of interest in step 3 of the procedure given following (8). Perhaps of greater historical interest, in the scalar case, are the results of Richardson and Wu (1971) who explore the properties of the relative bias. However, these results are of limited interest for two reasons. First, and probably most important, because the parameterisation of their model differs from that generated by the local-to-zero asymptotics, their tabulated results are not comparable with what is done here. Second, to reiterate, the relative bias is only of interest to us inasmuch as it allows us, given certain other information, to chose a non-centrality parameter useful in determining SY critical values in the scalar case. That is, unlike for all of the above-mentioned papers, moments are only of tangential interest to us, as this is not a study of the estimators themselves. To the best of our knowledge, Section 6 provides the first treatment of the (local-to-zero asymptotic) relative bias in the general case.

8. Conclusions

The main contribution of this paper has been to resolve analytically an integral as a special function, obviating the need to resolve it by simulation. This integral is of independent interest in the theory of ratios of quadratic forms in normal variables. Here, it is of primary interest because it provides a functional relationship between the bias in the 2SLS estimator and the limiting sampling distribution of a test statistic that SY proposed for testing the presence of weak instruments, when the null of weak instruments is true. Analysis of this special function provides theoretical foundations for the remarks of Section 3, which explore patterns observed in Table 1 as the parameters B and $k_z$ vary. This analysis required the derivation of certain results that are of independent interest in the theory of confluent hypergeometric functions. We have also explored the problem of p-values of the aforementioned test for weak instruments, on the basis of our earlier theoretical developments. We provide information such that any computer software that can then evaluate a non-central chi-squared cdf can readily compute p-values in essentially all practical circumstances. The final contribution of this paper has been the analysis of the general case characterised by an arbitrary number of endogenous regressors. Here, we find that the analysis is able to draw heavily on the foundations laid down in the literature on exact sampling distributions. This allows us to provide expressions for both the expectation of interest and also first- and second-order many-instrument expansions of this expectation. The exact expression obtained for the integral of interest in the general case is not of great practical interest, as it involves invariant polynomials with two matrix arguments for which, at the time of writing, there exist no algorithms for their computation, except in special cases. Nevertheless, the asymptotic expansions obtained are readily computable and potentially of practical importance. Given our reservations about the usefulness of the overall procedure in the general case, we leave such explorations to others.

One aspect of the SY tables that we have not addressed relates to those tables based on size distortions of a Wald statistic. This is a much more difficult analytical problem than has been addressed here and it is not clear that there is much benefit in tackling it as, in our estimation, the bias tables are in much more frequent use, making them of greater practical relevance.

Finally, in support of the results presented in the paper, we provide two Matlab programs on an ‘as is’ basis. The first of these, Table1.m, provides the body of Table 1. The second program, entitled sypval.m, provides p-values. Appendix C provides some discussion on the contents of these programs. The programs are available at https://sites.google.com/site/skeelscv/.