Multivariate Global-Local Priors for Small Area Estimation

Abstract

It is now widely recognized that small area estimation (SAE) needs to be model-based. Global-local (GL) shrinkage priors for random effects are important in sparse situations where many areas’ level effects do not have a significant impact on the response beyond what is offered by covariates. We propose in this paper a hierarchical multivariate model with GL priors. We prove the propriety of the posterior density when the regression coefficient matrix has an improper uniform prior. Some concentration inequalities are derived for the tail probabilities of the shrinkage estimators. The proposed method is illustrated via both data analysis and simulations.

1. Introduction

Small area estimation (SAE) has been gaining increasing popularity in recent years. Its need is felt both in the public and private sectors. By now, it is well recognized that SAE must be model-based due to the lack of sufficient samples in individual local areas. The most well-known area-level SAE model is due to Fay and Herriot [1], which essentially is a random effect model, each area level effect being treated as a random effect.

While the Fay–Herriot model has enjoyed wide popularity for nearly four decades, questions have been raised recently regarding the need to include random effects for all areas. Datta et al. [2] were the first to address this problem. They suggested a preliminary test-based approach where the null hypothesis was that the common random effect variance was zero. The test was based on a discrepancy statistic measuring the lack of fit of the fixed effect model. Their proposal was to use a fixed or a random effect model based on the acceptance or rejection of the null hypothesis. Further results in this direction are due to Molina et al. [3] and Morales et al. [4].

The method performs well when the number of small areas is moderately large, but, as often happens in practice, the number of small areas is very large, for example, when one considers all counties in the United States. In such situations, even if the regression estimates can describe the small area means very well in most of the small areas, the null hypothesis of no random effects is still very likely to be rejected. This phenomenon appears because, for a few areas, the direct estimates deviate from the regression estimates significantly, even after taking into account the large sampling errors. This problem was first realized by Datta and Mandal [5]. They proposed to model random effects through a mixture of a point mass at zero and a zero-mean normal distribution. Such priors belong to the general class of spike-and-slab priors. The point mass part is suitable for areas where fixed effect models are adequate, while the normal distribution part models random effects when this is not the case.

In contrast to the spike-and-slab priors of Datta and Mandal [5], Tang et al. [6] used global-local shrinkage priors for random effects in small area estimation, which captured wide area-level variation when the number of small areas was very large. These global-local priors employ two levels of parameters, global and local parameters, to express variances of area-specific random effects so that both small and large random effects can be captured properly. The global parameter causes shrinkage on all random effects to capture the small random effects, while the local parameters try to avoid over-shrinkage for areas that need large random effects. The degree of this neutralizing effect is closely related to the tails of the priors for the local parameters. If the prior is appropriately heavy-tailed, then both small and large random effects can be well captured. Moreover, one of the virtues of global-local priors is that they enable one to assess the individual area-level effects, rather than the blanket dichotomization of zero or non-zero effects.

The objective of this paper is to generalize the arguments of Tang et al. [6] to a multidimensional SAE model. There are many instances when one needs multivariate SAE models. A classic example is the simultaneous estimation of the median income of three-, four- and five-person families initiated in Fay [7], and followed up later by Ghosh et al. [8]. A second example is the estimation of unemployment rates, as considered in Datta et al. [9]. A third example is the adjustment of census undercounts considered in Datta et al. [10]. Unlike this paper, the above articles model the area-level effects with only one global variance matrix. In many cases, there is built-in dependence among the direct estimates for different components, which demands the multivariate model.

Returning to the present paper, a hierarchical prior is introduced in Section 2, where we have proven the propriety of the resulting posterior under some conditions. Section 3 discusses the implementation of the proposed method via Markov chain Monte Carlo, and several priors for local parameters are listed in Section 3.2. Concentration inequalities related to the tail behavior of the posterior are given in Section 4. A real data example is considered in Section 5, while Section 6 provides some simulation results. Final remarks are made in Section 7.

The global-local shrinkage priors were introduced in a series of articles by Carvalho et al. [11], Polson and Scott [12], Polson and Scott [13], Polson and Scott [14], Polson and Scott [15] and Scott [16]. They have been extended into a richer class. Some recent inventions are the three-parameter beta normal (TPBN) priors by Armagan et al. [17] and the generalized double Pareto (GDP) priors by Armagan et al. [18]. TPBN itself is a large class and includes the now famous horseshoe (HS) prior Carvalho et al. [11], the normal-exponential-gamma (NEG) priors Griffin and Brown [19] and the Strawderman–Berger (SB) priors (Strawderman [20], Berger [21]). These priors have been used successfully in multiple testing by Datta and Ghosh [22], Ghosh et al. [23], and also in other contexts. GL priors can be further classified into polynomial-tailed priors and exponential-tailed priors according to the tail behavior of the local parameters.

2. The Hierarchical Model

We begin with a multivariate analog of the model proposed in Tang et al. [6].

$$\begin{array}{cc}\hfill & {\mathit{y}}_i={\mathbf{\theta }}_i+{\mathit{e}}_i,{\mathbf{\theta }}_i=\mathit{B}{\mathit{x}}_i+{\mathit{u}}_i,{\mathit{e}}_i\sim {\mathcal{N}}_k(0,{\mathit{V}}_i),\pi \left(\mathit{B}\right)=1,\hfill \\ \hfill & {\mathit{u}}_i|{\lambda }_i^2,\mathbf{\Sigma }\sim {\mathcal{N}}_k(0,{\lambda }_i^2\mathbf{\Sigma }),{\lambda }_i^2\stackrel{iid}{\sim }{\pi }_{{\lambda }^2},\mathbf{\Sigma }\sim {\pi }_{\mathbf{\Sigma }},i=1, 2, \dots , m,\hfill \end{array}$$

(1)

where the direct survey estimators ${\mathit{y}}_i$, the corresponding means ${\mathbf{\theta }}_i$, the random effects ${\mathit{u}}_i$ and the error term ${\mathit{e}}_i$ are all k-dimensional vectors for the i-th area; $\mathit{B}$ is a $k\times p$ matrix of regression coefficients; and m is the number of small areas. The variance matrices ${\mathit{V}}_i$s are assumed to be known to avoid nonidentifiability. ${\lambda }_i^2$ are local parameters and $\mathbf{\Sigma }$ is a $k\times k$ matrix for the global parameter. All error terms ${\mathit{e}}_i$ and random effects ${\mathit{u}}_i$ are independent random variables. It is assumed that all the ${\mathit{V}}_i$ are positive definite. Moreover, the smallest eigenvalues of all the ${\mathit{V}}_i$ are bounded below by $v_m$, while the corresponding largest eigenvalues are bounded above by $v_M$, where $0<v_m\le v_M<\infty$.

We introduce the notations $\mathit{Y}={({\mathit{y}}_1, \dots , {\mathit{y}}_m)}^{\top }$, $\mathit{X}={({\mathit{x}}_1, \dots , {\mathit{x}}_m)}^{\top }$, $\mathit{e}={({\mathit{e}}_1, \dots , {\mathit{e}}_m)}^{\top }$, $\mathbf{\theta }={({\mathbf{\theta }}_1, \dots , {\mathbf{\theta }}_m)}^{\top }$ and $\mathit{u}={({\mathit{u}}_1, \dots , {\mathit{u}}_m)}^{\top }$. In matrix notations, $\mathit{Y}=\mathit{X}{\mathit{B}}^{\top }+\mathit{e}+\mathit{u}$.

Thus, $vec\left({\mathit{e}}^{\top }\right)={[e_{1,1}, \dots , e_{m,1}, e_{1,2}, \dots , e_{m,2}, \dots , e_{1,n}, \dots , e_{m,n}]}^{\top }\sim {\mathcal{N}}_{mk}(0,\mathit{V})$, where $\mathit{V}=diag({\mathit{V}}_1\dots {\mathit{V}}_m)$. We assume rank $\left(\mathit{X}\right)=p(\le m)$. Moreover, we will write ${\mathit{D}}_{\lambda }=diag({\lambda }_1^2, \dots , {\lambda }_m^2)$. Hence, $vec\left({\mathbf{\theta }}^{\top }\right)\sim {\mathcal{N}}_{mk}(vec\left({\left(\mathit{X}{\mathit{B}}^{\top }\right)}^{\top }\right),{\mathit{D}}_{\lambda }\otimes \mathbf{\Sigma })$. Since an improper prior density has been used for $\mathit{B}$, we need to find conditions under which the posterior density is proper. To this end, we first define a matrix normal distribution.

Definition 1.

A random matrix $\mathit{Z}$ is said to have the matrix-normal density if $\mathit{Z}$ has the density function (on the space ${\mathbb{R}}^{m\times k}):$

$$f\left(\mathbf{Z}\right)=\frac{exp\left(-\frac{1}{2}\mathrm{tr}\left[{\mathbf{W}}^{-1}(\mathbf{Z}-\mathbf{M}){\mathbf{R}}^{-1}{(\mathbf{Z}-\mathbf{M})}^{\top }\right]\right)}{{\left(2\pi \right)}^{mk/2}{\left|\mathbf{R}\right|}^{m/2}{\left|\mathbf{W}\right|}^{k/2}},$$

where $\mathbf{M}\in {\mathbb{R}}^{m\times k}$, and $\mathbf{W}$ and $\mathbf{R}$ are positive definite matrices of dimension $m\times m$ and $k\times k$, respectively. We write $\mathit{Z}\sim {\mathcal{MN}}_{m\times k}(\mathbf{M},\mathbf{W},\mathbf{R})$. Thus, the random effect matrix $\mathit{u}\sim {\mathcal{MN}}_{n\times k}(0,{\mathit{D}}_{\lambda },\mathbf{\Sigma })$.

We establish the propriety of the posterior under arbitrary proper priors for the local parameters ${\lambda }_i^2$ and the global parameter $\mathbf{\Sigma }$ in the following theorem, whose proof is given in Appendix A.1. In Section 3, we will have particular choices of priors for the ${\lambda }_i^2$ and $\mathbf{\Sigma }$ for actual implementation.

Theorem 1.

The posterior distribution corresponding to the (1) is proper if both ${\pi }_{{\lambda }_i^2}\left({\lambda }_i^2\right)$, $i=1, \dots , m$ and ${\pi }_{\mathbf{\Sigma }}\left(\mathbf{\Sigma }\right)$ are proper.

In the next section, we will consider several choices of priors for the local parameters ${\lambda }_i^2$. Throughout, for actual implementation, we will consider the inverse Wishart prior

$${\pi }_{\mathbf{\Sigma }}\left(\mathbf{\Sigma }\right)\propto {\left|\mathbf{\Sigma }\right|}^{-(1/2)({\nu }_0+k+1)}exp\left[\mathrm{tr}\left(\mathbf{\Psi }{\mathbf{\Sigma }}^{-1}\right)\right],$$

which we symbolically denote as ${\mathcal{W}}^{-1}(\mathbf{\Psi },{\nu }_0)$.

3. Computation and Local Prior Selection

3.1. Computation

In this section, we have derived the full conditionals for Gibbs sampling. It is convenient to compute all the conditionals in terms of the ${\mathbf{\theta }}_i$ rather than the ${\mathit{u}}_i$. Since ${\mathbf{\theta }}_i=\mathit{B}{\mathit{x}}_i+{\mathit{u}}_i$, which is a one–one linear transformation from $({\mathbf{\theta }}_i,\mathit{B})\mapsto ({\mathit{u}}_i,\mathit{B})$, and the Jacobian matrix is constant and depends only on $\mathit{X}$, we can rewrite the joint density of our model in (1) as

$$\begin{array}{c}\begin{array}{cc}\hfill \pi (\mathit{B},\mathbf{\Sigma },\mathbf{\theta },{\lambda }^2& \left|\mathit{Y}\right)\propto exp\left[-\frac{1}{2}\sum _{i=1}^m{({\mathit{y}}_i-{\mathbf{\theta }}_i)}^{\top }{\mathit{V}}_i^{-1}({\mathit{y}}_i-{\mathbf{\theta }}_i)\right]\frac{1}{{\left|\mathbf{\Sigma }\right|}^{m/2}{({\lambda }_1\cdots {\lambda }_m)}^k}\hfill \end{array}\hfill \\ \hfill \hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\times exp\left[-\sum _{i=1}^m\frac{1}{2{\lambda }_i^2}{({\mathbf{\theta }}_i-\mathit{B}{\mathit{x}}_i)}^{\top }{\mathbf{\Sigma }}^{-1}({\mathbf{\theta }}_i-\mathit{B}{\mathit{x}}_i)\right]{\pi }_{\mathbf{\Sigma }}\left(\mathbf{\Sigma }\right){\pi }_{{\lambda }_i^2}\left({\lambda }_1^2\right)\cdots {\pi }_{{\lambda }_m^2}\left({\lambda }_m^2\right).\end{array}$$

(2)

To compute the conditional distribution for $\mathit{B}$, we need to simplify the following expression:

$$\begin{array}{cc}\hfill \sum _{i=1}^m\frac{1}{{\lambda }_i^2}& {({\mathbf{\theta }}_i-\mathit{B}{\mathit{x}}_i)}^{\top }{\mathbf{\Sigma }}^{-1}({\mathbf{\theta }}_i-\mathit{B}{\mathit{x}}_i)=\sum _{i=1}^m\frac{1}{{\lambda }_i^2}tr\left({\mathbf{\Sigma }}^{-1}({\mathbf{\theta }}_i-\mathit{B}{\mathit{x}}_i){({\mathbf{\theta }}_i-\mathit{B}{\mathit{x}}_i)}^{\top }\right)\hfill \\ \hfill & =tr\left[{\mathbf{\Sigma }}^{-1}\sum _{i=1}^m\left(\frac{{\mathbf{\theta }}_i{\mathbf{\theta }}_i^{\top }}{{\lambda }_i^2}+\frac{\mathit{B}{\mathit{x}}_i{\mathit{x}}_i^{\top }{\mathit{B}}^{\top }}{{\lambda }_i^2}-\frac{\mathit{B}{\mathit{x}}_i{\mathbf{\theta }}_i^{\top }}{{\lambda }_i^2}-\frac{{\mathbf{\theta }}_i{\mathit{x}}_i^{\top }{\mathit{B}}^{\top }}{{\lambda }_i^2}\right)\right]\hfill \\ \hfill & =tr\left[{\mathbf{\Sigma }}^{-1}\left({\mathbf{\theta }}^{\top }{\mathit{D}}_{\lambda }^{-1}\mathbf{\theta }+\mathit{B}{\mathit{X}}^{\top }{\mathit{D}}_{\lambda }^{-1}\mathit{X}{\mathit{B}}^{\top }-\mathit{B}{\mathit{X}}^{\top }{\mathit{D}}_{\lambda }^{-1}\mathbf{\theta }-{\mathbf{\theta }}^{\top }{\mathit{D}}_{\lambda }^{-1}\mathit{X}{\mathit{B}}^{\top }\right)\right]\hfill \\ \hfill & =tr\left[{\mathbf{\Sigma }}^{-1}\left((\mathit{B}-\mathit{M})\left({\mathit{X}}^{\top }{\mathit{D}}_{\lambda }^{-1}\mathit{X}\right){(\mathit{B}-\mathit{M})}^{\top }+{\mathbf{\theta }}^{\top }({\mathit{D}}_{\lambda }^{-1}-{\mathit{P}}_{\lambda })\mathbf{\theta }\right)\right],\hfill \end{array}$$

(3)

where $\mathit{M}={\mathbf{\theta }}^{\top }{\mathit{D}}_{\lambda }^{-1}\mathit{X}{\left({\mathit{X}}^{\top }{\mathit{D}}_{\lambda }^{-1}\mathit{X}\right)}^{-1}$ and ${\mathit{P}}_{\lambda }={\mathit{D}}_{\lambda }^{-1}\mathit{X}{\left({\mathit{X}}^{\top }{\mathit{D}}_{\lambda }^{-1}\mathit{X}\right)}^{-1}{\mathit{X}}^{\top }{\mathit{D}}_{\lambda }^{-1}$.

Using Equation (3), we can rewrite (2) as

$$\begin{array}{cc}\hfill \pi & (\mathit{B},\mathbf{\Sigma },\mathbf{\theta },{\lambda }^2|\mathit{Y})\propto exp\left[-\frac{1}{2}\sum _{i=1}^m{({\mathit{y}}_i-{\mathbf{\theta }}_i)}^{\top }{\mathit{V}}_i^{-1}({\mathit{y}}_i-{\mathbf{\theta }}_i)\right]\frac{1}{{\left|\mathbf{\Sigma }\right|}^{m/2}{({\lambda }_1\cdots {\lambda }_m)}^k}{\pi }_{{\lambda }^2}\left({\lambda }_1^2\right)\cdots {\pi }_{{\lambda }^2}\left({\lambda }_m^2\right)\hfill \\ \hfill & \times exp[-\frac{1}{2}tr\left({\mathbf{\Sigma }}^{-1}\left((\mathit{B}-\mathit{M})\left({\mathit{X}}^{\top }{\mathit{D}}_{\lambda }^{-1}\mathit{X}\right){(\mathit{B}-\mathit{M})}^{\top }+{\mathbf{\theta }}^{\top }({\mathit{D}}_{\lambda }^{-1}-{\mathit{P}}_{\lambda })\mathbf{\theta }\right)\right]{\pi }_{\mathbf{\Sigma }}\left(\mathbf{\Sigma }\right)\hfill \end{array}$$

(4)

Using (2) and (4), one can obtain the Gibbs sampler designed to sample from the posterior density. Full conditional distributions are given below.

• ${\mathbf{\theta }}_i|\mathit{B},{\lambda }^2,\mathbf{\Sigma },\mathit{Y}\stackrel{ind}{\sim }{\mathcal{N}}_k({(\mathbf{I}+{\mathit{V}}_i{\mathbf{\Sigma }}^{-1}/{\lambda }_i^2)}^{-1}({\mathit{y}}_i+{\mathit{V}}_i{\mathbf{\Sigma }}^{-1}\mathit{B}{\mathit{x}}_i/{\lambda }_i^2),{(\mathbf{I}+{\mathit{V}}_i{\mathbf{\Sigma }}^{-1}/{\lambda }_i^2)}^{-1}{\mathit{V}}_i)$
• $\mathit{B}|\mathbf{\theta },{\lambda }^2,\mathbf{\Sigma },\mathit{Y}\sim {\mathcal{MN}}_{n\times k}(\mathit{M},\mathbf{\Sigma },{\left({\mathit{X}}^{\top }{\mathit{D}}_{\lambda }^{-1}\mathit{X}\right)}^{-1})$
• $\mathbf{\Sigma }|\mathbf{\theta },\mathit{B},{\lambda }^2,\mathit{Y}\sim {\mathcal{W}}^{-1}(\mathbf{\Psi }+{(\mathbf{\theta }-\mathit{X}{\mathit{B}}^{\top })}^{\top }{\mathit{D}}_{\mathbf{\lambda }}^{-1}(\mathbf{\theta }-\mathit{X}{\mathit{B}}^{\top }),m+{\nu }_0)$

To complete the computation, in the next section, we introduce several widely used local priors, and then discuss some specifics regarding their implementation.

3.2. Local Prior Selection

We have provided a short list of widely used priors for the local parameters in

Table 1. Global-local shrinkage prior for ${\mathit{u}}_i$.

Prior	$\mathit{\pi }\left({\mathit{\lambda }}_{\mathit{i}}^2\right)$	Class
Laplace (LA)	$exp(-{\lambda }_i^2)$	E
Strawderman–Berger (SB)	${(1+{\lambda }_i^2)}^{-3/2}$	P
Horseshoe (HS)	${\lambda }_i^{-1}{(1+{\lambda }_i^2)}^{-1}$	P
Normal-exponentail-gamma (NEG)	${(1+{\lambda }_i^2)}^{-(1+b)}$	P
Normal beta prime (NBP)	${\left({\lambda }_i^2\right)}^{a-1}{(1+{\lambda }_i^2)}^{-(a+b)}$	P

The first column gives the names of priors of u_i marginalized over λ²_i. The second column gives the corresponding priors of the local parameters λ²_i up to a normalizing constant. In the third column, E and P stand for exponential-tailed priors and polynomial-tailed priors, respectively.

. The priors $\pi \left({\mathit{u}}_i\right|\mathbf{\Sigma })$ are derived after marginalizing out the local parameters ${\lambda }_i^2$.

To facilitate the discussion of computation, we further group the priors in Table 1 into generalized inverse Gaussian priors and normal beta prime priors.

• Generalized Inverse Gaussian (GIG): ${\lambda }_i^2\stackrel{ind}{\sim }GIG(\delta ,\chi ,\psi )$ if ${\pi }_{\lambda }^2\left({\lambda }_i^2\right)\propto {\lambda }_i^{2(\delta -1)}exp(-\chi /2{\lambda }_i^2-\psi {\lambda }_i^2/2)$. Thus, ${\lambda }_i^2|\mathbf{\theta },\mathit{B},\mathbf{\Sigma },\mathit{Y}\stackrel{ind}{\sim }GIG(\delta -k/2,\chi +{({\mathbf{\theta }}_i-\mathit{B}{\mathit{x}}_i)}^{\top }{\mathbf{\Sigma }}^{-1}({\mathbf{\theta }}_i-\mathit{B}{\mathit{x}}_i),\psi )$. Using this, we can see that if ${\lambda }_i^2$ is exponential, viz. ${\pi }_{\lambda }^2\left({\lambda }_i^2\right)\propto exp(-{\lambda }_i^2)$, then ${\lambda }_i^2|\mathbf{\theta },\mathit{B},\mathbf{\Sigma },\mathit{Y}\stackrel{ind}{\sim }GIG(1-k/2,{({\mathbf{\theta }}_i-\mathit{B}{\mathit{x}}_i)}^{\top }{\mathbf{\Sigma }}^{-1}({\mathbf{\theta }}_i-\mathit{B}{\mathit{x}}_i),2)$. The Laplace prior is a special case of GIG priors with $\delta =1$, $\chi =0$ and $\psi =2$.
• Normal Beta Prime (NBP): ${\pi }_{{\lambda }^2}\left({\lambda }_i^2\right)\propto {\lambda }_i^{2(a-1)}{(1+{\lambda }_i^2)}^{-a-b}, a>0, b>0$
  One obtains NBP prior ${\pi }_{{\lambda }^2}\left({\lambda }_i^2\right)\propto {\lambda }_i^{2(a-1)}{(1+{\lambda }_i^2)}^{-a-b}$ by introducing a latent variable ${\xi }_i\sim Gamma(b,1)$ and $\pi \left({\lambda }_i^2\right|{\xi }_i)\propto {\left({\lambda }_i^2\right)}^{a-1}{\xi }_i^{a}exp(-{\xi }_i{\lambda }_i^2)$. Therefore, ${\lambda }_i^2|\mathbf{\theta },\mathit{B},\mathbf{\Sigma },\xi ,\mathit{Y}\stackrel{ind}{\sim }GIG(a-k/2,{({\mathbf{\theta }}_i-\mathit{B}{\mathit{x}}_i)}^{\top }{\mathbf{\Sigma }}^{-1}({\mathbf{\theta }}_i-\mathit{B}{\mathit{x}}_i),2{\xi }_i)$ and ${\xi }_i|\mathbf{\theta },\mathit{B},\mathbf{\Sigma },\lambda ,\mathit{Y}\stackrel{ind}{\sim }Gamma(a+b,{\lambda }_i^2+1)$. By setting different values for a and b, we will obtain Strawderman–Berger (SB), horseshoe (HS) and negative exponential gamma (NEG) as special cases. For NEG, we choose b = 0.75 in our Data Analysis section.

We will study Laplace and normal beta prime priors in Section 4 to illustrate our theoretical results. For the simulation study and real data analysis, we will use Laplace (LA), Strawderman–Berger (SB), horseshoe (HS) and negative exponential gamma (NEG, $b=0.75$) as local priors in our multivariate models.

4. Shrinkage Factor

Despite their distinct forms, these multivariate GL shrinkage priors possess a common feature—the ability to assign nontrivial probability mass both near zero and in the tail—which enables our multivariate GL model to capture both small and large random effects based on data. To see this, first note that, given the local and global parameters, the conditional posterior mean of the small area mean ${\mathbf{\theta }}_i$ shrinks the direct estimate ${\mathit{y}}_i$ toward the synthetic regression estimate $\mathit{B}{\mathit{x}}_i$ as

$$\begin{array}{c}\hfill E\left({\mathbf{\theta }}_i\right|\mathit{B},{\lambda }^2,\mathbf{\Sigma },\mathit{Y})={(I+{\mathit{V}}_i{\mathbf{\Sigma }}^{-1}/{\lambda }_i^2)}^{-1}({\mathit{y}}_i+{\mathit{V}}_i{\mathbf{\Sigma }}^{-1}\mathit{B}{\mathit{x}}_i/{\lambda }_i^2),\end{array}$$

(5)

where ${\mathit{S}}_{GL,i}={(I+{\mathit{V}}_i{\mathbf{\Sigma }}^{-1}/{\lambda }_i^2)}^{-1}{\mathit{V}}_i{\mathbf{\Sigma }}^{-1}/{\lambda }_i^2$ is called a shrinkage factor. A larger (smaller) shrinkage factor causes more (less) shrinkage and produces an estimate closer to the synthetic estimate (direct estimate). Here, ${\parallel .\parallel }_F$ denotes the Frobenius norm of a matrix in this section.

Theorem 2.

Suppose $\pi \left({\lambda }_i^2\right)$ is a proper pdf with support $(0,\infty )$. Then, for $0<ϵ<1$,

$$\begin{array}{c}\hfill P(\parallel {\mathit{S}}_{GL,i}{\parallel }_F<ϵ)\le \frac{|{\mathit{V}}_i{+g\mathbf{\Sigma }|}^{1/2}}{|{\mathit{V}}_i{|}^{1/2}}exp\left[{({\mathit{y}}_i-\mathit{B}{\mathit{x}}_i)}^{\top }{\mathit{V}}_i^{-1}({\mathit{y}}_i-\mathit{B}{\mathit{x}}_i)/2\right]\frac{{\int }_{c/{\parallel \mathbf{\Sigma }\parallel }_F}^{\infty }\pi \left({\lambda }_i^2\right)d{\lambda }_i^2}{{\int }_0^g\pi \left({\lambda }_i^2\right)d{\lambda }_i^2},\end{array}$$

(6)

where $c=k(\frac{1}{ϵ}-1)\frac{1}{\parallel {\mathit{V}}_i^{-1}{\parallel }_F}$ and $g(>0)$ an arbitrary positive constant.

The proof of the theorem is provided in Appendix A.2. This theorem leads immediately to the result $P(\parallel {\mathit{S}}_{GL,i}{\parallel }_F<ϵ)\to 0$ as ${\parallel \mathbf{\Sigma }\parallel }_F\to 0$. We now illustrate this result with some of the global-local priors.

Example 1.

Consider the Laplace prior $\pi \left({\lambda }_i^2\right)=exp(-{\lambda }_i^2)$. Then,

$$\frac{{\int }_{c/{\parallel \mathbf{\Sigma }\parallel }_F}^{\infty }\pi \left({\lambda }_i^2\right)d{\lambda }_i^2}{{\int }_0^g\pi \left({\lambda }_i^2\right)d{\lambda }_i^2}=\frac{exp(-c/\parallel \mathbf{\Sigma }{\parallel }_F)}{1-exp(-g)},$$

which converges to zero at an exponential rate when ${\parallel \mathbf{\Sigma }\parallel }_F\to 0$.

Example 2.

Consider the NEG prior $\pi \left({\lambda }_i^2\right)\propto {\left({\lambda }_i^2\right)}^{a-1}{(1+{\lambda }_i^2)}^{-a-b}$. Then, for $a\ge 1$ and by variable transformation $u=\frac{{\lambda }_i^2}{1+{\lambda }_i^2}$, we obtain

$${\int }_{c/{\parallel \mathbf{\Sigma }\parallel }_F}^{\infty }\pi \left({\lambda }_i^2\right)d{\lambda }_i^2={\int }_{c/(\parallel \mathbf{\Sigma }{\parallel }_F+c)}^1{u}^{a-1}{(1-u)}^{b-1}du/B(a,b)\le \frac{1}{b}{\left[\frac{{\parallel \mathbf{\Sigma }\parallel }_F}{{\parallel \mathbf{\Sigma }\parallel }_F+c}\right]}^b/B(a,b)$$

which again converges to zero as ${\parallel \mathbf{\Sigma }\parallel }_F\to 0$. For $0<a<1$,

$$\begin{array}{c}\hfill {\int }_{c/{\parallel \mathbf{\Sigma }\parallel }_F}^{\infty }\pi \left({\lambda }_i^2\right)d{\lambda }_i^2\le {\left[\frac{c}{{\parallel \mathbf{\Sigma }\parallel }_F+c}\right]}^{a-1}\frac{1}{b}{\left[\frac{{\parallel \mathbf{\Sigma }\parallel }_F}{{\parallel \mathbf{\Sigma }\parallel }_F+c}\right]}^b\to 0\mathrm{as}{\parallel \mathbf{\Sigma }\parallel }_F\to 0.\end{array}$$

Theorem 3.

Suppose that the prior $\pi \left({\lambda }_i^2\right)$ has support $(0,\infty ).$ Then, for $ϵ>0$, $P(\parallel {\mathit{S}}_{GL,i}{\parallel }_F>ϵ|\mathit{B},\mathbf{\Sigma },{\mathit{y}}_i)\le K_i\frac{{\int }_0^{c_1/s_m}\pi \left({\lambda }_i^2\right)d{\lambda }_i^2}{{\int }_0^{c_{1}d/s_M}\pi \left({\lambda }_i^2\right)d{\lambda }_i^2}$, where $c_1=\sqrt{k}{\parallel {\mathit{V}}_i\parallel }_F/ϵ$, $d(>0)$ is a positive constant and $K_i=|{\mathit{V}}_i+c_{1}d{\mathbf{I}}_k{|}^{1/2}{\left|{\mathit{V}}_i\right|}^{-1/2}exp\left[(1/2){({\mathit{y}}_i-\mathit{B}{\mathit{x}}_i)}^{\top }{\mathit{V}}_i^{-1}({\mathit{y}}_i-\mathit{B}{\mathit{x}}_i)\right]$, where $s_m$ and $s_M$ are the smallest and largest eigenvalues of Σ, respectively.

The proof of the theorem is provided in Appendix A.3.

Remark 1.

It is only by proper choice $d(>0)$ that we obtain $P(\parallel {\mathit{S}}_{GL,i}{\parallel }_F>ϵ)\to 0$ as $\parallel {\mathbf{\Sigma }}^{-1}{\parallel }_F\to 0$ in the examples to consider. Moreover, we need to assume that $\underset{s_m\to \infty }{lim\; sup}{s}_M/s_m<\infty$ or, alternatively, we can rewrite this assumption as $\underset{\parallel {\mathbf{\Sigma }}^{-1}{\parallel }_F\to 0}{lim\; sup}\frac{\mathrm{Largest}\mathrm{eigen}\mathrm{value}\mathrm{of}\mathbf{\Sigma }}{\mathrm{Smallest}\mathrm{eigen}\mathrm{value}\mathrm{of}\mathbf{\Sigma }}<\infty$.

We now apply Theorem 3 for Examples 1 and 2.

Example 3.

This example is the continuation of Example 1. The ratio $\frac{{\int }_0^{c_1/s_m}\pi \left({\lambda }_i^2\right)d{\lambda }_i^2}{{\int }_0^{c_{1}d/s_M}\pi \left({\lambda }_i^2\right)d{\lambda }_i^2}$ simplifies to $\frac{1-exp(-c_{1}M/s_m)}{1-exp(-c_{1}dM/s_M)}<exp$$\left[c_{1}M(d/s_M-1/s_m)\right]$, which converges to zero for $d>w$ as $\parallel {\mathbf{\Sigma }}^{-1}{\parallel }_F\to 0$, where $w=\underset{\parallel {\mathbf{\Sigma }}^{-1}{\parallel }_F\to 0}{lim\; sup}{s}_M/s_m$.

Example 4.

This example is the continuation of Example 2. $\frac{{\int }_0^{c_1/s_m}\pi \left({\lambda }_i^2\right)d{\lambda }_i^2}{{\int }_0^{c_{1}d/s_M}\pi \left({\lambda }_i^2\right)d{\lambda }_i^2}=R$, say, where $R=\frac{{\int }_0^{c_1/(s_m+c_1)}{u}^{a-1}{(1-u)}^{b-1}du}{{\int }_0^{c_{1}d/(s_M+c_{1}d)}{u}^{a-1}{(1-u)}^{b-1}du}$. For $b\ge 1$,

$$\begin{array}{c}\hfill R\le \frac{{\left(\frac{c_1}{s_m+c_1}\right)}^a}{{\left(\frac{c_{1}d}{s_M+c_{1}d}\right)}^a{\left(\frac{s_M}{s_M+c_{1}d}\right)}^{b-1}}=d^{-a}\frac{{\left(\frac{s_M/s_m+c_{1}d/s_m}{1+c_1/s_m}\right)}^a}{{\left(\frac{s_M}{s_M+c_{1}d}\right)}^{b-1}}\end{array}$$

Now, for $b<1$,

$$\begin{array}{c}\hfill R\le \frac{{\left(\frac{c_1}{s_m+c_1}\right)}^a{\left(\frac{s_m}{s_m+c_1}\right)}^{b-1}}{{\left(\frac{c_{1}d}{s_M+c_{1}d}\right)}^a}=d^{-a}{\left(\frac{s_M/s_m+c_{1}d/s_m}{1+c_1/s_m}\right)}^a{\left(\frac{s_m}{s_m+c_1}\right)}^{b-1}\end{array}$$

Hence, $\underset{\parallel {\mathbf{\Sigma }}^{-1}{\parallel }_F\to 0}{lim\; sup}R<d^{-a}{w}^a$, where $w=\underset{\parallel {\mathbf{\Sigma }}^{-1}{\parallel }_F\to 0}{lim\; sup}{s}_M/s_m$. For any arbitrarily small δ, choose $d(>0)$ such that $d^{-a}{w}^a<\delta$. Therefore, $P(\parallel {\mathit{S}}_{GL,i}{\parallel }_F>ϵ|\mathit{B},\mathbf{\Sigma },{\mathit{y}}_i)\to 0$ as $\parallel {\mathbf{\Sigma }}^{-1}{\parallel }_F\to 0$.

Theorem 4.

Suppose ${\pi }_{{\lambda }_i^2}\left({\lambda }_i^2\right)$ is a proper prior with support on $(0,\infty )$; then,

$$P(\parallel {\mathit{S}}_{GL,i}{\parallel }_F>ϵ|\mathit{B},\mathbf{\Sigma },{\mathit{y}}_i)\le c_{3}exp(-d\parallel {\mathit{y}}_i-\mathit{B}{\mathit{x}}_i{\parallel }_2^2/2)$$

, where $c_3$ and d are some positive finite constants.

The proof of Theorem 4 is given in Appendix A.4 and this theorem immediately gives us $P(\parallel {\mathit{S}}_{GL,i}{\parallel }_F>ϵ|\mathit{B},\mathbf{\Sigma },{\mathit{y}}_i)\to 0$ as $\parallel {\mathit{y}}_i-\mathit{B}{\mathit{x}}_i{\parallel }_2\to \infty$.

5. Data Analysis

We consider the problem of estimating median income. The Census Bureau releases estimates of median household income for many different demographic and geographic subgroups. A model-based approach is needed to obtain finer breakdowns (by demographics and/or geography) while still maintaining adequate precision. We applied our method to estimate a four-dimensional response, which is the median income of the homeowner, renter, married and unmarried populations. These groups are determined by the head of the household, and some of these groups overlap each other, e.g., a homeowner could be both married and an owner. The dataset was compiled using the one-year public-use microsample from the 2015 American Community Survey (data available at https://data.census.gov, accessed on 1 May 2019). The direct estimates are obtained at state level, including the District of Columbia. Thus, the number of small areas m is 51. The descriptive statistics are presented in

Table 2. Descriptive statistics for sample sizes and median household income across the 51 states/districts.

	Unweighted Sample Size			Median Household Income (in Thousands)
	Min	Median	Max	Min	Median	Max
Homeowner	1254	12,587	79,099	49.6	66.0	125.0
Renter	533	4942	53,101	23.0	33.7	55.0
Married	804	8245	67,913	62.6	78.0	167.3
Unmarried	1121	8842	64,287	25.0	35.4	62.0

. Besides the intercept, per-capita income is included as the covariate (data available at https://bea.gov, accessed on 1 May 2019). Moreover, we have scaled down the values of the data by dividing both the direct estimates and the covariate by 1000. The error variance–covariance matrices, ${\mathit{V}}_i,i=1, \dots , m$, are also rescaled accordingly.

In our data analysis, we consider model (1) with five different choices of the priors on the variance–covariance matrix of ${\mathit{u}}_i$. In the first model, we assume ${\lambda }_i^2=1$ for $i=1, \dots , m$. Thus, the random effects ${\mathit{u}}_i$ share a common variance–covariance matrix $\mathbf{\Sigma }$ across different small areas. It is essentially a multivariate Fay–Herriot model. The remaining four models vary in the prior of the local parameters ${\lambda }_i^2$. We consider the LA, HS, SB, and NEG priors, which are stated in Table 1. Among the four choices, HS, SB, and NEG are special cases of the normal beta prime family and have polynomial tails. In all five models, we used an inverse-Wishart prior for $\mathbf{\Sigma }$ with degrees of freedom ${\nu }_0=4$ and $\mathbf{\Psi }=\mathit{I}$. For each of these models, we run the Gibbs sampler described in Section 3 for 20,000 iterations. The first half is discarded as the burn-in period. The samples from the remaining 10,000 iterations are used for inference. The small area means ${\mathbf{\theta }}_i$ and the random effects ${\mathit{u}}_i$ are estimated by the corresponding posterior sample means.

We first examine the estimated random effects from different models. Figure 1 presents the posterior means of random effects ${\mathit{u}}_i$. Since the results from the three models with polynomial-tailed local priors are similar, we only present the results for HS as a representative. For most of the states, the estimated random effects from different models do not differ much. However, significant differences are seen in the District of Columbia, especially for the married and the renter groups. There is also a visible difference in Connecticut for the renter and the unmarried groups. When there is a difference, the estimated random effects from the LA model are usually smaller than those from the HS model and larger than those from the FH model. This observation demonstrates that polynomial-tailed priors lead to less shrinkage towards zero when compared with exponential-tailed priors.

Figure 2 presents the estimated ${\mathbf{\theta }}_i$ from two GL models (LA and HS) against those from the FH model. For most of the states, the estimates agree for all four subgroups. For the District of Columbia, the estimates for the married group from the GL models, especially the HS model, are higher than those from the FH model. It is the only state or district where the estimated ${\mathbf{\theta }}_i$ from the HS model differs from that from the FH model by more than 5%. It is not surprising that DC differs from the rest of the states since this area is more similar to large urban counties than other states, but it is typically included in state-level small area models for completeness of the entire U.S. population. The flexibility in our GL model is able to account for the difference through a larger random effect.

To select an appropriate model for the dataset, we use the deviation information criterion (DIC) proposed by Spiegelhalter et al. [24]. The simulation study in Section 6 shows that the model selected by DIC and the model producing the lowest deviation measurements often give comparable estimates of small area means. For the real median income dataset, the DICs for the FH, LA, HS, SB, and NEG models are 1018.30, 967.79, 957.10, 957.46, and 959.48, respectively. The HS model has the smallest DIC and is thus selected.

Table 3. Median income estimates from the HS model.

State	Owner	Renter	Married	Unmarried	State	Owner	Renter	Married	Unmarried
AL	55.79	25.09	70.67	27.19	MT	62.03	30.45	73.01	31.41
AK	87.77	41.12	100.78	44.22	NE	70.85	32.35	83.23	34.72
AZ	63.63	35.54	73.97	35.29	NV	68.89	37.03	75.50	37.11
AR	52.84	26.72	64.51	27.01	NH	84.95	39.72	96.88	42.73
CA	87.40	44.45	90.00	45.00	NJ	96.17	42.20	108.18	45.30
CO	79.88	39.90	89.78	41.52	NM	55.43	28.28	67.90	29.10
CT	95.05	38.14	110.37	44.20	NY	83.20	38.44	93.06	40.00
DE	74.53	37.40	88.15	39.53	NC	60.77	30.89	73.22	31.12
DC	125.03	51.60	156.71	58.82	ND	78.48	36.20	90.61	38.94
FL	59.88	35.00	71.90	33.98	OH	65.40	30.08	78.85	32.36
GA	65.53	33.01	76.09	33.49	OK	60.03	30.91	72.09	31.13
HI	91.35	54.73	96.93	51.29	OR	69.74	35.10	77.84	36.01
ID	59.06	30.16	69.10	30.44	PA	69.46	32.22	83.10	35.00
IL	75.32	34.92	89.03	37.73	RI	81.99	33.38	94.95	38.46
IN	62.11	29.87	74.99	31.70	SC	57.13	29.98	71.68	30.03
IA	66.10	30.08	78.82	32.68	SD	65.60	29.80	76.58	32.14
KS	67.87	33.02	80.82	34.22	TN	59.81	29.03	71.28	30.00
KY	56.08	27.99	68.47	29.01	TX	70.95	36.93	80.00	37.01
LA	59.97	25.97	76.61	28.17	UT	75.70	39.27	79.22	40.03
ME	62.31	29.56	75.48	31.59	VT	73.98	35.65	85.58	37.96
MD	96.19	47.15	111.49	50.03	VA	81.65	41.86	94.87	41.96
MA	94.90	40.19	109.58	44.97	WA	80.49	41.32	90.06	41.89
MI	62.18	29.96	76.59	31.90	WV	50.74	24.61	64.58	26.10
MN	78.30	35.41	91.69	39.50	WI	69.78	32.87	81.35	35.26
MS	49.92	24.02	64.42	25.20	WY	73.63	40.26	83.75	39.52
MO	62.62	30.18	75.03	31.89

gives the estimated median income from the HS model for the four subgroups in each state.

6. Simulation Study

In this section, we investigate the performance of different priors for estimating the small area means on simulated datasets. In the simulation study, data are generated according to ${\mathit{y}}_i={\mathbf{\theta }}_i+{\mathit{e}}_i$, where ${\mathit{e}}_i\sim {\mathcal{N}}_k(0,{\mathit{V}}_i)$ and ${\mathbf{\theta }}_i=\mathit{B}{\mathit{x}}_i+{\mathit{u}}_i$ for $i=1, \dots , m$. We first consider a data generation setting similar to the real dataset in Section 5. We set $m=51$ and $k=4$. The per-capita income (pci) from the real dataset is used as the covariate. The $\mathit{X}={(x_1, \dots , x_m)}^{\top }$ matrix consists of centered pci and a column of ones, so $p=2$. The coefficient matrix $\mathit{B}$ is set at the least square estimate computed from the real data. The error variance matrices ${\mathit{V}}_i, i=1, \dots , m$ are also borrowed from the real dataset. Three models are considered for generating ${\mathit{u}}_i$:

• common variance: ${\mathit{u}}_i\sim {\mathcal{N}}_k(0,\mathbf{\Sigma })$ with the element in the j-th row and l-th column of $\mathbf{\Sigma }$ being ${0.3}^{|j-l|}$;
• two-component: ${\mathit{u}}_i\sim {\mathcal{N}}_k(0,{\lambda }_i^2\mathbf{\Sigma })$, where ${\lambda }_i^2$ is 0.01 or 10 with equal probability and $\mathbf{\Sigma }$ is the same as in the common variance model;
• multi-variance: ${\mathit{u}}_i\sim {\mathcal{N}}_k(0,{\lambda }_i^2\mathbf{\Sigma })$, where ${\lambda }_i^2, i=1, \dots , m$ are equally spaced values from 0.01 to 10 and $\mathbf{\Sigma }$ is the same as in the common variance model.

To investigate the performance when the number of small areas is large, we consider two additional choices of m, 500 and 1000. For both choices, k, p, and $\mathit{B}$ are the same as in the previous setting. The $\mathit{X}$ matrix contains a column of ones and a column of values generated from $\mathcal{N}(0,{\sigma }^2)$, where ${\sigma }^2=68.65$ is the sample variance of pci in the real dataset. For each small area, ${\mathit{V}}_i$ is randomly sampled from the error variance matrices in the real dataset. The three models for generating ${\mathit{u}}_i$ in the setting $m=51$ are also used in the settings of $m=500$ and 1000.

We generate 100 datasets for each combination of m and the model for ${\mathit{u}}_i$. For each dataset, the five models considered in Section 5 are fitted. For each model, the Gibbs sampler is run for 20,000 iterations. The samples from the first 10,000 iterations are discarded. Then, ${\theta }_i$ is estimated by the posterior sample mean computed from the remaining 10,000 samples. The estimation accuracy is evaluated using four deviance measures, average absolute deviation (AAD), average squared deviation (ASD), average absolute relative deviation (ARB), and average squared relative deviation (ASRB), which are defined as follows:

$$\begin{array}{c}\mathrm{AAD}=\frac{1}{mk}{\displaystyle \sum _{i=1}^m}{\displaystyle \sum _{j=1}^k}|{\widehat{\theta }}_{ij}-{\theta }_{ij}|,\mathrm{ASD}=\frac{1}{mk}{\displaystyle \sum _{i=1}^m}{\displaystyle \sum _{j=1}^k}{({\widehat{\theta }}_{ij}-{\theta }_{ij})}^2,\\ \mathrm{ARB}=\frac{1}{mk}{\displaystyle \sum _{i=1}^m}{\displaystyle \sum _{j=1}^k}|{\widehat{\theta }}_{ij}-{\theta }_{ij}|/{\theta }_{ij},\mathrm{ASRB}=\frac{1}{mk}{\displaystyle \sum _{i=1}^m}{\displaystyle \sum _{j=1}^k}{({\widehat{\theta }}_{ij}-{\theta }_{ij})}^2/{\theta }_{ij}^2.\end{array}$$

(7)

The averages of each deviation measure across 100 datasets are presented in Figure 3. Under the common variance setting, the FH model produces the smallest estimation errors in terms of all four measures. This is expected because the fitted model is the same as the data generation model. Although the GL models do not match the data generation model, their performance is similar to the FH model.

Under the other two settings for ${\mathit{u}}_i$, the GL models often have the best performance. Especially under the two-component setting, the FH model produces much higher deviation measures compared to GL models. Although the results from different GL models do not vary much under the two settings, the HS, SB, and NEG models have better performance than the LA model under the two-component setting, while the reverse is true under the multi-variance setting. The difference in the performance is caused by the tail of the prior on the local parameters. Polynomial-tailed priors often shrink the signals less than the exponential-tailed priors, so the models with polynomial-tailed priors (HS, SB, NEG) tend to perform better in the two-component case, where the signals (large random effects) and the noises (small random effects) are well separated.

We also investigate the performance of the model selected according to DIC. The results are also plotted in Figure 3. As the figure shows, the deviation measurements produced by the model selected by DIC are often close to the lowest among all models in each setting. This observation indicates that DIC usually selects a reasonable model for a given dataset.

7. Final Remarks

In this paper, we have addressed the situation of multivariate sparsity in random effects. We are able to estimate the random effects in income for different correlated groups and the correlation structure is used in the estimation process. We are able to derive concentration inequalities of the tail behavior of shrinkage factors. Our simulation study shows that the proposed multivariate GL estimators are very close to the truth. The data analysis shows that our multivariate GL model can identify the states/districts that have significantly large random effects.

Our simulation study shows that the prior that works best varies across data generation settings. Overall, priors with polynomial tails perform better when the small and large random effects are well separated, while the priors with exponential tails perform better when there are many intermediate random effects. We demonstrate that DIC can be used to select a suitable prior for a dataset.

There is potential for generalizing these results. In particular, in the proposed model, the same local parameter is assumed for different components of a small area. To account for heterogeneous dependence among components across areas, one can assume the area-level variance matrix for area i to be ${\sigma }^2{\Gamma }_i$ instead of ${\lambda }_i^2\mathbf{\Sigma }$. Moreover, one can address the same problem for unit-level models with global-local priors involving two variance components for area-level effects.

Moreover, independent random effects are considered in the proposed model. Recently, random effects with spatial or spatio-temporal dependence have been considered in small area estimation when suitable covariates for accounting for dependence in the geographic or time domain are not available (cf. Chung and Datta [25], Bradley et al. [26]). Combining the global-local shrinkage prior with spatial or spatio-temporal priors is also a potential future direction.