Modeling Community Health with Areal Data: Bayesian Inference with Survey Standard Errors and Spatial Structure

Abstract

Epidemiologists and health geographers routinely use small-area survey estimates as covariates to model areal and even individual health outcomes. American Community Survey (ACS) estimates are accompanied by standard errors (SEs), but it is not yet standard practice to use them for evaluating or modeling data reliability. ACS SEs vary systematically across regions, neighborhoods, socioeconomic characteristics, and variables. Failure to consider probable observational error may have substantial impact on the large bodies of literature relying on small-area estimates, including inferential biases and over-confidence in results. The issue is particularly salient for predictive models employed to prioritize communities for service provision or funding allocation. Leveraging the tenets of plausible reasoning and Bayes’ theorem, we propose a conceptual framework and workflow for spatial data analysis with areal survey data, including visual diagnostics and model specifications. To illustrate, we follow Krieger et al.’s (2018) call to routinely use the Index of Concentration at the Extremes (ICE) to monitor spatial inequalities in health and mortality. We construct and examine SEs for the ICE, use visual diagnostics to evaluate our observational error model for the ICE, and then estimate an ICE–mortality gradient by incorporating the latter model into our model of sex-specific, midlife (ages 55–64), all-cause United States county mortality rates. We urge researchers to consider data quality as a criterion for variable selection prior to modeling, and to incorporate data reliability information into their models whenever possible.

1. Introduction

Community survey data has long served as an important source of evidence in epidemiology. Du Bois’s, The Philadelphia Negro [1,2], among the first modern social epidemiological studies [3], combined original household survey data with United States (U.S.) Census data and municipal health reports to evaluate the role of social conditions, rather than purported biological traits, as determinant of differences in mortality rates among Black and White Philadelphians. Similarly, census tract-level indicators of social class have long substituted for individual-level data in analyses of health inequality [4,5]. Today, epidemiologists and health geographers routinely employ community-level survey data in models of health outcomes because they have access to a variety of geographic data products, including neighborhood-level information. Unfortunately, excitement over access to ‘fine-grained’ geospatial data has generally not been tempered by a realistic assessment of the tradeoffs between data granularity and data quality. For commonly used survey data products such as the American Community Survey (ACS), one should expect data quality to deteriorate as one moves toward smaller spatial scales and toward more detailed concepts and demographic breakdowns. Likewise, one would expect data reliability to vary across demographics and places as a function of social integration/marginalization because standard errors (SEs) are largely a result of sample size and survey response rates. In other words, data quality may often be correlated with the very community characteristics that are of primary interest to investigators. Similar concerns hold for (non-survey based) raster data products, such as Earth Institute’s high-spatial-resolution Gridded Population of the World products, notably their sex-specific five-year age group population estimates [6]. This challenge is distinct from, and compounds with, the challenge of inferring underlying patterns of risk from limited observation, such as occurs when big data is disaggregated into small geographic areas and/or multiple demographic groups (see [7]).

This paper proposes a conceptual framework and workflow to support population health research with areal survey data, including visual diagnostics and model specifications. Section 2 provides background information on ACS methodology, including the Census Bureau’s systematic spatial sampling design. We examine a purposeful sample of variables to illustrate how SEs accompanying estimates vary systematically across regions and neighborhoods, by demographic characteristics, and by survey topic (cf. [8,9]). We then review the impacts that sampling error from ACS products may have on descriptive statistics and on inferences when survey estimates are used as covariates [10]. We extend established results on measurement error to a spatial context, arguing that observational error with spatial data has the additional impact of tending to conceal spatial autocorrelation (SA), and, by implication, inflating effective sample size [11,12]. Modeling spatial data with observational error may be a more perilous endeavor than existing research on measurement error alone would imply. Section 3 draws on the findings of the previous sections, as well as previous research on hierarchical Bayesian models (HBMs) for spatial data [13,14,15,16,17,18,19,20], to develop a methodology suitable for modeling community survey data (cf. [19,21]). Appendix A serves as an introduction to plausible reasoning with HBMs, detailing a framework for conceptualizing and building inferential models that incorporate observational uncertainty.

We demonstrate the proposed workflow while building on Krieger, Kim, Feldman, and Waterman’s [22] call to routinely use the Index of Concentration at the Extremes (ICE) [23] to monitor spatial inequalities in health and mortality. We evaluate the reliability of ICE and population at risk data, summarize prior findings on county-level social class–mortality gradients, and then model U.S. county-level, all-cause, sex- and age-specific (55–64 years) mortality rates, comparing results from a “naive” model with our proposed HBM methodology. We urge researchers to routinely incorporate SA and data reliability information into their research workflow, from study design to model criticism and reporting.

2. The American Community Survey

The ACS, including the Census Bureau’s subsidiary Puerto Rico Community Survey, is the largest and surely the most widely used source of small-area survey data in the U.S. In accordance with sound scientific practice, the Census Bureau reports estimates together with their SEs whenever possible. These SEs remain woefully underappreciated by the scientific community, in part because of their recent appearance. This section reviews the ACS methodology, examines systematic patterns in the SEs of select ACS variables (extending [8,9]), and discusses some of the implications for models that include ACS estimates as covariates.

2.1. A Systematic Spatial Sampling Design

ACS is a continually operating survey of U.S. households that began operations in 2005, and in 2011 increased its target annual sample size from 2.9 to 3.54 million households and increased in-person follow-up rates for non-responding households to $100\%$ in select low-population and primarily Indigenous communities [24] (Ch. 4). The sampling design is systematic by block group (see [25], pp. 23–43, on spatial sampling), and blocks with lower estimated population and lower expected response rates are sampled at higher rates to protect the quality of estimates [24] (Ch. 4). Published estimates are the product of a multi-stage weighting and (for missing, highly implausible, and inconsistent responses) imputation process, and are harmonized with the Census Bureau’s population estimates by sex, age, race, Hispanic origin, and total household units. The three- and five-year estimates are the sum of all weighted responses from the preceding n years of surveying [24] (Ch. 11). ACS estimates are accompanied by margins of error ($90\%$ confidence intervals) that “reflect the variation in the estimates over all possible samples that could have been selected from the population using the same sampling methodology” not inclusive of possible recording errors and explicitly excluding possible biases in the sampling design [24] (Ch. 12). The Census Bureau calculates margins of error for ACS estimates using the Successive Differences Replication (SDR) method [26,27,28]. This methodology involves repeated sampling from sub-sets of the weighted observations, and then calculating the mean squared error of the replicate estimates from the observed estimate. These procedures can be applied to any function of the estimates such as the ratio, sum, or difference of any two variables. The Census Bureau provides variance replicate tables for a limited number of variables, so that users can calculate SEs for composite variables of their own construction (e.g., deprivation indices) [29].

2.2. ACS Standard Errors

This section identifies prominent patterns in SEs for select ACS variables—percent over age 24 with a bachelors degree or higher, median household (HH) income, and percent with health insurance—at the U.S. county level ($n=3142$) and at the census-tract level for a single county, Milwaukee, Wisconsin ($n=296$). These are ACS 2018 5-year estimates. The first two of these variables were selected because they are commonly used as indicators of social class, and the third variable was chosen for its particular relevance to public health research. Sampling variability is a function of both sample size (and thus, by extension, survey response rates) and population heterogeneity [8]. Neither response rates nor area heterogeneity are independent of sociopolitical factors, such as social marginalization and patterns of spatial segregation. The findings presented here establish the following characteristics within our sample of variables:

• SEs contain systematic spatial patterns;
• Data reliability is not constant across variables;
• Data describing affluent areas is often of higher reliability than that of impoverished areas; and,
• Data describing areas estimated to be majority Black, Hispanic, or American Indian tends to be lower quality than that of majority White areas.

Points 1 and 3 are strikingly clear from the maps of ACS data quality in Figure 1 and Figure 2. For example, counties in the northeastern region are estimated to have high median HH incomes and low coefficients of variation ($\mathrm{CV}=\frac{\mathrm{SE}}{\mathrm{Estimate}}$), whereas Appalachia and the Deep South have low estimated incomes and relatively high CVs. Analogous spatial patterns appear across impoverished and affluent areas of Milwaukee County. These observations are consistent with previous findings [8,9], including, for example, that ACS estimates of median Black HH income “are especially bad for the poorest 15 percent of census tracts” [8] (p. 152).

Table 1. Degree of SA in select ACS estimates and their respective measures of data reliability (CV for income, SEs for the others).

	U.S. Counties			Milwaukee County Census Tracts
	Income	Insurance (%)	College Ed. (%)	Income	Insurance (%)	College Ed. (%)
black Estimates	0.80	0.76	0.69	0.84	0.82	0.93
Data reliability	0.56	0.69	0.57	0.46	0.59	0.52

Note: The values reported are estimates of the SA parameter $\rho$ from an intercept-only SAR model, which range from $\frac{1}{{\lambda }_{min}}$ to 1, where ${\lambda }_{min}$ is the most negative eigenvalue of matrix W (and, ${\lambda }_{max}=1$).

quantifies the degree of SA in each variable and in their respective measures of reliability (SE or CV) using the SA parameter from an intercept-only simultaneous autoregressive (SAR) model. The auto-normal SAR specification models n observations of an outcome, y, using the multivariate normal distribution as the likelihood function, instead of specifying the likelihood as the product of n independent, univariate normal distributions. The covariance matrix, $\mathsf{\Sigma }$, incorporates a row-standardized spatial connectivity matrix, $\mathit{W}$, with parameter $\rho$ accounting for the nature and degree of SA, and $\sigma$ for the usual scale parameter: $\mathsf{\Sigma }={\sigma }^2{\left[(\mathit{I}-\rho {\mathit{W}}^{\prime })(\mathit{I}-\rho \mathit{W})\right]}^{-1}$ [20] (pp. 198–200). The SEs of all three variables at both scales show moderately high levels of SA (from 0.46 to 0.69), similar to many other socioeconomic and demographic variables.

Table 2. Relative magnitude of the median data quality measure (CVs for income, SEs for the others) by ACS variable, geography, and majority racial-ethnic group.

	Income	Insurance (%)	College Ed. (%)
	U.S. Counties
White	1	1	1
Black	2	1.36	1.17
Hispanic	1.67	1.72	1.11
Native American	1.67	1.85	1.22
	Milwaukee County Census Tracts
White	1	1	1
Black	1.625	1.5	0.85
Hispanic	1.38	2.39	0.59
Native American	-	-	-

Note: Black, White, and Native American all indicate non-Hispanic identifying populations.

reports relative data quality (as measured by their median value) by majority racial-ethnic group, with majority White areas serving as the reference category; Figure 3 plots data quality against estimates themselves. We see that all three variables tend to be of higher quality in majority White areas and affluent areas, with the exception of educational attainment in Milwaukee, for which the inverse holds. This is almost certainly related to the degree of homogeneity of the population—standard errors are smallest in census tracts where the proportion of college graduates is nearest to zero. Similarly, the SEs for health insurance coverage shrink as the estimates approach one. This is consistent with the formula for the sampling variance of a proportion, p, from the binomial distribution, $VAR\left(p\right)=p\ast (1-p)/n$, where n is the sample size. Figure 4 shows that data quality can be highly variable across survey topics—estimates of tract-level insurance coverage are of substantially inferior quality than those of educational attainment. Notice, many of the SEs are greater than the median absolute deviation (MAD) of the variable itself. These findings indicate that data quality is itself a social variable.

2.3. Implications

The inter-related, systematic patterns present in ACS data quality may impact inferences in complex ways. Past findings regarding measurement error provide a useful starting point, if only to indicate the minimal severity of the problem. Sampling error is most appropriately modeled following the framework of “classical” measurement error, which represents observations, $\mathit{X}$, as the sum of the actual values, $\mathcal{X}$, and errors, $\mathbf{\Delta }$,

$$\mathit{X}=\mathcal{X}+\mathbf{\Delta }$$

(1)

where each is an $n\times k$ matrix, where n is the number of areas observed and k is the number of variables. Three important and well-known results follow from this representation [10] (pp. 1–64):

• Additive error tends to increase sample variance, leading to exaggerated confidence in regression estimates;
• the additional variability tends to attenuate bivariate correlations and mask non-linear relations; and,
• in multivariate models, observational error may result in a change of sign, attenuation, or exaggeration of coefficient estimates at any sample size.

Figure 5 illustrates what is meant by a tendency toward attenuation in bivariate regression—observed relations may still be attenuated or exaggerated, but with large numbers of observations, attenuation becomes a near certainty and confidence intervals become wildly misleading (cf. [30,31]). With spatial data, such as small-area ACS estimates, two additional insights follow directly from Equation (1):

4.

Observational error tends to decrease the degree of observed SA; and,

5.

Spatial variation in data quality tends to produce spatial variation in the analytical consequences of observational error.

The latter observation has implications for models that attempt to infer spatially varying relations among variables, such as geographically weighted regression [32]. Geographic variation in the corruption of observations may produce spurious geographic variation in relations between variables (geographically varying bias). The former observation indicates that our measures of SA may often be underestimated, which implies an over-estimation of effective sample size or, to rephrase, an improper weighting of evidence.

3. Spatial HBMs for Survey Data

This section presents our proposed methodology for modeling small-area survey data. We illustrate and validate the analytical argument by comparing rival models for a single variable, health insurance coverage, and we provide a set of visual diagnostic plots to evaluate our model. We then use our methodology to compare raw ACS estimates for select variables, $\mathit{X}$, with the probability distributions for their respective true values, $\mathcal{X}$, noting inferential problems that may be introduced by the confluence of SA and observational error.

3.1. Prior Information and Model Specification

HBMs are built by successive application of the product rule for expressing the joint probability of multiple propositions (see Appendix A for additional details). Bayes’ theorem provides a method for calculating the probability of a proposition given data, X, and any relevant (prior) information, I. The probability of the proposition, after seeing the data, is known as the ‘posterior probability’, leading to the summary expression of Bayes’ theorem,

$$\mathrm{Posterior}\propto \mathrm{Likelihood}\times \mathrm{Prior}.$$

(2)

When considering observational error, or complex spatial or spatio-temporal information, the joint probability expands into numerous terms, leaving Equation (2) wanting for clarity. Thus Clayton [13,14,15,16] proposed to factor epidemiological HBMs into,

$$\mathrm{Posterior}\propto [\mathrm{Measurement}\mathrm{model}]\times [\mathrm{Disease}\mathrm{model}]\times [\mathrm{Exposure}\mathrm{model}],$$

(3)

whereas Berliner [17,18,20] introduced the generic terms,

$$\mathrm{Posterior}\propto [\mathrm{Data}\mathrm{model}]\times [\mathrm{Process}\mathrm{model}]\times [\mathrm{Parameter}\mathrm{model}].$$

(4)

In both cases, the concept is identical: the first term incorporates information about the measurement or observational process that may have introduced a difference between the state of reality, $\mathcal{X}$, and our record of it, $\mathit{X}$. The process model encodes substantive knowledge of the process under study, and the parameter model encodes contextual knowledge about the possible states of that process.

Building HBMs for ACS data requires specification of three probability models: the data/measurement model, $p\left(\mathit{X}\right|\mathcal{X},I)$, the process model, $p\left(\mathcal{X}\right|\mathit{\zeta },I)$ where $\mathit{\zeta }$ are process parameters, and the parameter/exposure model, $p\left(\mathit{\zeta }\right|I)$. For the first, we assign to the errors, $\mathbf{\Delta }$, a Gaussian probability density with variance, ${\sigma }^2$, equal to the square of the SEs of the estimates, ${\mathit{S}}^2$, such that $\mathit{X}\sim Gauss(\mathcal{X},\mathit{S})$. This specification may be justified for continuous variables by the maximum entropy principle insofar as our information for each observation consists of only location (estimate) and scale (SE) parameters [33,34]. The Census Bureau’s practice of calculating 90% margins of error by multiplying the SEs by 1.645 also implies a Gaussian probability distribution for the sampling errors. For the process model, $p\left(\mathcal{X}\right|\mathit{\zeta },I)$, we require a probability model that incorporates the most pertinent and generalizable information we have about socioeconomic and health variables. Foremost are the following social and economic characteristics of contemporary society:

• Polarization, such that relatively extreme values are not unexpected; and,
• Segregation, such that most social and economic variables display moderate to strong SA.

The first postulate conflicts with the Gaussian model, which places very low probability on outliers and extreme values. The second observation eliminates the uniform distribution from consideration, because it would prevent us from incorporating SA. The uniform distribution would also result in models that place high probability on values that are implausibly far from the range of observed estimates, given that ACS SEs can be quite large (polarization does not imply unrestricted variation).

An auto-Gaussian model that incorporates SA in the covariance matrix automatically increases the probability of tail-area (extreme) values relative to an independent Gaussian model, while also placing low probability on outliers relative to the local area mean. This model incorporates both postulates simultaneously because outliers (relative to the global mean) tend to cluster together at commonly employed units of aggregation (census tracts, counties, and states). Similar to Kang, Liu, and Cressie [19], we suggest the following model specification for small-area survey data:

$$\begin{array}{cc}\hfill [\mathrm{Data}\mathrm{model}]:\mathit{X}& \sim Gauss(\mathcal{X},\mathit{S})\hfill \\ \hfill [\mathrm{Process}\mathrm{model}]:\mathcal{X}& \sim MVGauss(\mathit{1}\mu ,{(\mathit{I}-\rho \mathit{W})}^{-1}\mathit{M})\hfill \\ \hfill [\mathrm{Parameter}\mathrm{model}]:\mu & \sim Gauss(\ast ,\ast )\hfill \\ \hfill \tau & \sim Studen{t}^{+}(\ast ,\ast ,\ast )\hfill \\ \hfill \rho & \sim Uniform(\frac{1}{{\lambda }_{min}},\frac{1}{{\lambda }_{max}}).\hfill \end{array}$$

(5)

The process model for $\mathcal{X}$ is an auto-Gaussian model with a conditional autoregressive (CAR) specification of the covariance matrix [20] (pp. 167–203) where $\mu$ is a constant mean multiplied by an $n\times 1$ vector of ones, $\mathit{M}={\tau }^2{\mathit{D}}^{-1}$ is a diagonal matrix of conditional variances consisting of the inverse of the number of neighbors of each respective areal unit $D_{i,i}^{-1}$ times a scale parameter, ${\tau }^2$, $\rho$ is a parameter accounting for the nature and degree of SA, and $\mathit{W}$ is a row-standardized connectivity matrix with zeroes on the diagonal (also given as data). $\mathit{W}$ is specified such that any element $W_{i,j}$ equals $D_{i,i}^{-1}$ if the i$th$ and j$th$ observations are neighbors, and zero otherwise (This is just one valid specification of the CAR model. For others, see [20,35]). We define neighbors using the queen contiguity condition [35] (p. 89), [36]. The range of permissible values for the SA parameter $\rho$ is determined by the smallest and largest eigenvalues (${\lambda }_{min}^{-1}$, ${\lambda }_{max}^{-1}$) of the matrix ${\mathit{M}}^{-1/2}\mathit{W}{\mathit{M}}^{1/2}$ [20]. The * symbol indicates prior parameters to be specified relative to the problem at hand. Finally, the models must reflect any natural boundaries in the data (e.g., percentages range only from zero to one hundred). Such truncated distributions are easily programmed into MCMC algorithms by placing boundaries on the parameter space. We experimented with one alternative specification: a Student’s t model with spatially varying mean, using eigenvector spatial filtering [37,38]. We conclude that the auto-Gaussian model performs similarly to the spatial t model, but with substantial computational advantages—it is fairly efficient with moderately large n ($n\approx 3000$) using Stan [39].

3.2. Model Evaluation

Here we model percent of residents in Milwaukee County census tracts who have health insurance (ACS variable DP03_0096P, see Figure 2c,d). As noted, this variable has large SEs, which renders results particularly sensitive to model specification error. We leverage this fact to highlight the differences between models. We compare results from the auto-Gaussian specification of Equation (5) with a non-spatial Gaussian model. For each model, we examine the differences, $\widehat{\mathbf{\Delta }}$, between the mean of their respective posterior distributions, $p\left(\mathcal{X}\right|\mathit{X},\mathit{S},I)$, and their raw ACS estimates. In other words, ${\widehat{\Delta }}_i$ is the mean of the posterior distribution of the error, ${\Delta }_i=X_i-{\mathcal{X}}_i$. Diagnostic plots reveal that the non-spatial model produces systematically biased inferential patterns, but results from the auto-Gaussian model appear reasonable.

Each panel in Figure 6 contains three diagnostic plots. The top figure is a point-interval plot of the raw ACS estimates against a summary of their respective posterior distributions (mean and $95\%$ credible intervals (CI)), highlighting that the non-spatial model imposes unidirectional shrinkage toward the global mean value on the estimates. Below the point-interval plot is a Moran scatter plot (see [25,35,40]), which reveals that the non-spatial model has moderately strong SA in its $\widehat{\mathbf{\Delta }}$ values. The map of $\widehat{\mathbf{\Delta }}$ reveals that a distinct sociospatial pattern underlies the SA—the majority Black and Hispanic inner city tracts all have ${\widehat{\Delta }}_i>0$. The auto-Gaussian model, by contrast, pulls estimates with large SEs toward the local mean, and, as a result, $\widehat{\mathbf{\Delta }}$ reveals no conspicuous or concerning spatial pattern. Because the spatial model incorporates additional information relative to the non-spatial model (and the degree of SA in these particular variables is strong), the posterior distributions tend to be narrower, reflecting a greater degree of confidence in results.

3.3. Examining Implications

Here we report differences in the mean, dispersion, and SA between the raw ACS point estimates and their posterior distributions obtained employing the proposed CAR model. Our concern is that using raw ACS variables as covariates is leading researchers to become overly confident in their model results, due to a combination of inflated sample variance and deflated SA.

We utilize ACS data on three variables for Milwaukee County census tracts to fit the model, and we report summary statistics in

Table 3. Summary statistics for select ACS variables, Milwaukee County census tracts, comparing raw ACS data to the posterior distribution of the CAR data model for the same variables.

		Mean	Standard Deviation	SA ($\mathit{\rho }$)
Log-income	ACS	10.69	0.49	0.88
	Model	10.7 [10.68, 10.71]	0.46 [0.45, 0.48]	0.90 [0.88, 0.91]
Insurance (%)	ACS	91.96	5.67	0.82
	Model	92.81 [92.52, 93.09]	4.59 [4.22, 4.96]	0.89 [0.86, 0.92]
College (%)	ACS	28.31	20.87	0.93
	Model	28.1 [27.69, 28.51]	20.63 [20.23, 21.02]	0.94 [0.93, 0.95]

Note: The model results are summarized here by their posterior means and $95\%$ CIs. The SA parameter $\rho$ is from an intercept-only SAR model; we obtained a posterior distribution for this value by employing the same method used for the mean and standard deviation, i.e., the summary statistic was calculated for each of $M=7500$ samples drawn from the joint posterior distribution $p\left(\mathcal{X}\right|\mathit{X},\mathit{S},I)$, resulting in a vector of M values proportional to the posterior distribution of the summary statistic. Each vector is summarized here by its mean and quantiles.

. Median HH income was log-transformed to better suit our model, and its SEs were appropriately transformed as well. (SEs for $log\left(x_i\right)$ may be approximated by the transformation $s_{x_i}\to \frac{1}{x_i}{s}_{x_i}$. We applied a simple Monte Carlo method to construct SEs for the transformed variate.) For each variable, the mean, standard deviation, and degree of SA $\rho$ (obtained from a SAR model) were calculated for each sample from the joint posterior distribution of parameters. For example, the posterior distribution for the standard deviation of tract-level, log-transformed median HH income has a mean of 0.46, with a $95\%$ CI of $[0.45,0.48]$, compared to a raw value of 0.49. Only small differences appear between the raw and modeled values of percent college educated. Yet the posterior distribution for insurance coverage has markedly greater SA ($\widehat{\rho }=0.89$, 95% CI: $[0.86,0.92]$) than the raw ACS estimate ($\widehat{\rho }=0.82$), and its standard deviation decreased by 20%—from 5.67 to 4.59 (CI: $[4.22,4.96]$); hence, a variance decline of ≈34%, from 32 to 21.

The amount of meaningful information present in the raw estimates of percent insured was twice inflated—once by sampling error directly, and again by the obfuscation of SA. Consider the concept of effective sample size $n^{\ast }$, the number of equivalent independent observations required to obtain the same information content as an autocorrelated sample [11] (p. 15). For data well-modeled by the Gaussian distribution, $\rho =0.82$ converts a nominal sample of $n=296$ to $n^{\ast }\approx 23$, whereas a value of $\rho =0.9$ has $n^{\ast }\approx 12$ [12] (Equation (3)). Hence, strong SA in covariates causes measures of uncertainty to deteriorate in quality, whether they be p-values or Bayesian CIs, with or without a spatial model describing the outcome variable [37,38,41,42]. Thus, the results presented in this section provide additional motivation to carefully evaluate data quality as a criterion for variable selection, and to properly model both SA and observational error.

4. Modeling U.S. County Mid-Life Mortality

In this section, we model all-cause sex-specific U.S. county mortality rates for ages 55–64, and estimate the ICE–mortality gradient following Krieger et al. [22]. We exclude Alaska due to substantial differences in the state’s county equivalents. Approximately half of Alaska’s area is a single county equivalent, larger in area that any other state, which introduces the modifiable areal unit problem, in its worst possible materialization. If our primary purpose in this paper were to model U.S. mortality rates, then we would incorporate Alaska’s county equivalents through, perhaps, its own independent model [43]. Using ACS SEs and our data model, we find that the quality of county-level ICE estimates is neither particularly poor nor negligible. We compare results from our proposed spatial HBM of mortality rates to a naive model that does not consider observational uncertainty but is otherwise identical. We find that the naive model underestimates the ICE–mortality gradient and generally produces more narrow CIs for the county mortality rates. Some counties have sizable differences in predicted mortality rates while in dozens of counties the increase in posterior uncertainty is substantial.

4.1. Data and Prior Information

We gathered county-level all-cause mortality and population-at-risk data from CDC Wonder by sex for ages 55–64, aggregating over years 2014 through 2018 (Figure 7a,b) [44]. We dropped counties for which the mortality data is censored, and we also dropped one area that is missing an ICE value. Thus, our analysis includes $n=2984$ counties with male mortality data, and $n=2875$ counties with female mortality data. We manually updated the connectivity structure to link together some rural, low-population counties with missing observations between them (see online supplementary material for additional information). We used the Census Bureau’s variance replicate tables [29] to calculate the ICE by county with appropriate SEs (Figure 7c,d). The ICE is calculated as

$$\mathrm{ICE}=\frac{\mathrm{No}.\mathrm{highest}\mathrm{income}\mathrm{households}-\mathrm{No}.\mathrm{lowest}\mathrm{income}\mathrm{households}}{\mathrm{Total}\mathrm{no}.\mathrm{households}},$$

(6)

with threshold incomes for lowest and highest income groups set to $\$20,000$ and $\$125,000$, respectively (following Krieger et al. [22]). The ICE ranges from −1 to 1, with 1 corresponding to a population where all HH incomes are $\ge \$125,000$ and −1 corresponding to a population where all HH incomes are <$20,000. Figure 8a plots the SEs of each ICE estimate divided by the MAD of the ICE itself (excluding counties with censored mortality data). The median SE is $0.17$ times the MAD, and $50\%$ of the SEs are between 0.11 and 0.24 times the MAD; the largest SE is 0.78 times the MAD. Figure 9 provides diagnostics for our auto-Gaussian data model for the ICE. The ${\widehat{\Delta }}_i$ values are not particularly large and have no SA, which is reassuring. Examination of CVs for the estimated population-years at risk shows that the vast majority of CVs are <0.05 (Figure 8b,c).

To gather prior information on the ICE–mortality gradient, we searched PubMed for published research containing “all cause mortality” or “premature mortality” as well as “county” in its title or abstract ((((“all cause mortality” [Title/Abstract])) OR (“premature mortality” [Title/Abstract])) AND (“county” [Title/Abstract])). Of 310 results, 24 appeared potentially relevant and were selected for closer inspection. Eight of these studies reported findings on the degree of inequality in county mortality rates, although no two studies employed the same measurement of inequality (see

Table A1. Summary of select published findings on inequality in all-cause county mortality.

Source	Outcome	Period	Comparison	Findings
[47]	Age-adjusted mortality, 0–19 yo.	1968–1992	Bottom v. top quintile by deprivation index	y. 1992 RR = 1.52; RR range: 1.45–1.65
[48]	Age-standardized mortality, 0–65 yo.	1960–2002	Bottom v. top quintile of median HH income	RR = 1.6 [1.6, 1.7]
[45]	Age- and sex-adjusted mortality	1988–1992	Lowest inequality, highest income v. highest inequality, lowest income; by quartile of each measure	RR = 1.22
[50]	Mortality, 15–24 yo.	1999–2007	Bottom v. top decile by deprivation index	$\mathrm{Male}\mathrm{RR}=1.90$$[1.87,1.94]$; Female RR  = 1.62 [1.56, 1.67]
[51]	Age-adjusted mortality	1999–2013	Highest v. lowest of 8 county classes, grouped by mortality	RR increased each year from 2.1 (1999) to 2.7 (2013)
[49]	Age-standardized mortality, 25–64 yo.	2000–2003	Quintiles of education (% bachelors degree+) and median HH income	Education RR = 1.64; Income RR = 1.78
[46]	Age-adjusted mortality, 0–75 yo.	2002–2006	Bottom v. top quartile of median HH income	RR = 1.41
[52]	Mortality, 45–64 yo.	2005–2009	Metropolitan areas with <5% poverty vs. non-metro. areas with >20% poverty	RR = 2.22 [2.20, 2.24]

Note: All reported uncertainty intervals are 95% confidence intervals. Findings based on the same mortality data as the present study are excluded from this summary to avoid allowing our data to inform our prior probability model.

). We measure inequality by the relative index of inequality (RII): the mortality rate ratio of the most disadvantaged over the most advantaged group. RII maintains conceptual consistency across rate ratios by keeping the most disadvantaged group in the numerator. Thus the quintile-based RII (${\mathrm{RII}}_5$) with counties ordered by mortality rates is $\frac{p80}{p20}$, whereas the comparable quantity for counties ordered by the ICE is $\frac{p20}{p80}$. Two studies comparing the bottom to the top quartile of counties (${\mathrm{RII}}_4$) ordered by socioeconomic variables [45,46] found ${\mathrm{RII}}_4$ values of 1.22 and 1.41, respectively; those comparing the first to fifth quintiles of counties (${\mathrm{RII}}_5$) ordered by socioeconomic variables [47,48,49] found ${\mathrm{RII}}_5$ between 1.5 and 1.8; and, those reporting RIIs by decile [50] or other tail-area grouping [51,52] found RIIs between 1.6 and 2.7.

Based on this semi-formal review, we expect ${\mathrm{RII}}_5$—comparing the 20th percentile ($p20$) to the 80th percentile ($p80$) of counties ordered by the ICE—to be greater than unity, and we would be surprised if it were larger than 2.2 (for further discussion, see Appendix B). Our exploratory analysis of the data, including the scatter plots of the ICE against log-mortality rates by Census region in Figure 7e,f, indicate that log-mortality rates show an approximately linear relationship with the ICE. This result means that any value of ${\mathrm{RII}}_5$ can be converted into its corresponding coefficient $\beta$ from a log-linear model: $\beta =\frac{d\log \left(y\right)}{dx}=\frac{\log \left({\mathrm{RII}}_5^{-1}\right)}{p80-p20}$ (see Figure 10a). The value $dx=p80-p20$ can be calculated from our ICE data model, and equals $0.179$$[0.176,0.182]$. To encode our substantive prior information about ${\mathrm{RII}}_5$ into a probability distribution for $\beta$, we use the following model:

$$\begin{array}{c}\hfill {\mathrm{RII}}_5\sim Gauss(1.6,0.3)\\ \hfill \beta =\frac{\log \left({\mathrm{RII}}_5^{-1}\right)}{0.179}.\end{array}$$

(7)

Figure 10b is a density plot of the Gaussian prior on ${\mathrm{RII}}_5$, whereas Figure 10c shows how that density transfers to values of $\beta$. Most of the probability density is assigned to values of $\beta$ between −4.5 and −1. We do not convert our continuous measure of the ICE into discrete quintiles or other bins before modeling because that modification would arbitrarily delete data in our possession, and, as Figure 11 illustrates, our observational uncertainty regarding ICE values implies considerable uncertainty regarding to which quintile many observations belong. The number of observations with ambiguous membership in either the first or the fifth quintile, say those having a probability between 0.2 and 0.8 of belonging to either one, is 443 or 15% of our observations on male mortality.

4.2. Process and Parameter Models

Because the mortality data, $\mathit{Y}$, for each respective sex consists of a vector of counts of a rare outcome (relative to the size of the population at risk), we assign a Poisson probability distribution to the likelihood, with mean and variance equal to the parameter $\mathit{\mu }=\mathit{\lambda }·\mathit{P}$, the elementwise product of rates, $\mathit{\lambda }$, and population-years at risk, $\mathit{P}$. We model male and female mortality independently, applying the same model specification to each. The logs of the male and female mortality rates show moderately strong SA, both with Moran coefficients (MC) of $\mathrm{MC}=0.56$. The ICE ($\mathcal{X}$) also has strong SA, with $\mathrm{MC}=0.643$$[0.638,0.648]$. We model SA in the outcome using the Besag-York-Mollié (BYM) specification [53,54]. Whereas the CAR model from Equation (5) combines spatial trends ($\rho \mathit{W}$) and independent variation ($\mathit{M}$) into a single covariance matrix, the BYM model achieves computational efficiency by splitting these components of the model into two separate parameter models. This separation requires two parameter vectors: an SA term, $\mathit{\varphi }$, plus the non-spatial term, $\mathit{\theta }$. The intrinsic CAR (ICAR) prior is placed on $\mathit{\varphi }$, where the SA parameter $\rho$ is implicitly fixed to 1; hence it places high prior probability on smooth variation. Setting $\rho =1$ also renders the joint probability distribution of $\mathit{\varphi }$ improper, in the sense that it does not integrate to one. This is addressed by constraining the values of $\mathit{\varphi }$ to sum to zero (see [35], pp. 246–247). We implement this constraint following Morris et al.’s [54,55] method.

The BYM model captures additional variation around the spatial trend by assigning a Gaussian prior with unknown scale to $\mathit{\theta }$. The relative influence of the two terms is controlled by their respective scale parameters, ${\tau }_{\varphi }$ and ${\tau }_{\theta }$. We include a separate intercept for each fully connected component of the graph structure embedded in $\mathit{W}$ [56], meaning that the continental U.S. and Hawaii (represented by dummy variables $\mathit{A}$), respectively, have their own intercepts ($\mathit{\alpha }={[{\alpha }_1,{\alpha }_2]}^{\prime }$). We also follow Freni-Sterrantino, Ventrucci, and Rue [56] in adjusting the scale of each connected component to render the prior distributions for scale parameters approximately equivalent across any valid spatial connectivity structure [57]. The ICAR model assigned to $\mathit{\varphi }$ is, effectively, two separate models, one for the counties of the continental U.S., and another for Hawaii, each with its own scale parameter (${\mathit{\tau }}_{\mathit{\varphi }}=[{\tau }_{{\varphi }_1}$, ${\tau }_{{\varphi }_2}$]). To model SA in our covariate [37] (pp. 10–18), [42,58,59], we add its mean spatially-lagged value, $\mathit{W}\mathcal{X}$, as an additional covariate with coefficient $\gamma$ (recall that $\mathit{W}$ is row-standardized) [38]. Our data and process models for county mortality rates are as follows:

$$\begin{array}{cc}\hfill [\mathrm{Data}\mathrm{model}]:\mathit{X}& \sim Gauss(\mathcal{X},\mathit{S})\hfill \\ \hfill [\mathrm{Process}\mathrm{model}]:\mathit{Y}& \sim Poiss(\mathit{\lambda }·\mathit{P})\hfill \\ \hfill \log \left(\mathit{\lambda }\right)& =\mathit{A}\mathit{\alpha }+\mathit{\varphi }+\mathit{\theta }+\gamma \mathit{W}\mathcal{X}+\beta \mathcal{X}\hfill \\ \hfill \mathcal{X}& \sim MVGauss(\mathit{1}\mu ,{(\mathit{I}-\rho \mathit{W})}^{-1}\mathit{M})\hfill \end{array}$$

(8)

We mean-center the ICE so that the intercepts $\mathit{\alpha }$ represent the mean log-mortality rates for their respective geographic areas. The remainder of the parameter model is diffuse or weakly informative [60] (p. 19), [61] (p. 55) relative to natural constraints on the data values (e.g., the ICE, and hence its mean $\mu$, is between −1 and 1) or substantive limitations (e.g., the mean log-mortality rate ${\alpha }_1$ must be negative):

$$\begin{array}{cc}\hfill {\alpha }_1& \sim Gauss(-5,5)\hfill \\ \hfill {\alpha }_2& \sim Gauss(0,5)\hfill \\ \hfill \gamma & \sim Gauss(0,5)\hfill \\ \hfill \mathit{\varphi }& \sim ICAR\left({\mathit{\tau }}_{\mathit{\varphi }}\right)\hfill \\ \hfill \mathit{\theta }& \sim Gauss(0,{\tau }_{\theta })\hfill \\ \hfill \mu & \sim Gauss(0,0.5)\hfill \\ \hfill {\tau }_{{\varphi }_1},{\tau }_{{\varphi }_2},{\tau }_{\theta },{\tau }_{\mathcal{X}}{]}^{\prime }& \sim Gaus{s}^{+}(0,1)\hfill \end{array}$$

(9)

Note that the prior for ${\alpha }_1$ is essentially uniform over the full range of possible values for the mean log-county mortality rate. We also compare results from the full HBM as specified above to a naive model that has the same specification except for the replacement of raw ACS estimates $\mathit{X}$ for the CAR data model of $\mathcal{X}$.

Other valid model specifications are available. The BYM component of the model, in particular, is widely used in the literature, mainly because it tends to be more efficient than using a proper CAR model. Our spatial connectivity matrix was built using an adjacency structure, and was supplemented by manual adjustments for certain low-population areas with missing neighboring observations. A downside of the adjacency method is that it may induce ‘information sharing’ and, potentially, ‘smoothing’ over neighboring observations that are highly dissimilar in terms of demographics [35] (pp. 245–249). For example, it may be undesirable to specify the same degree of connectedness between a majority Native American county and its neighboring, majority White counties, as one might specify between any number of majority White counties, because it conflicts with our knowledge that such populations are not subject to similar sociopolitical conditions. If the purpose of a model is to determine health service provision, for example, then such a choice could have detrimental (and unwarranted) impacts. Our model is primarily for demonstration purposes, and we emphasize that many other modeling purposes require closer attention be paid to such questions.

4.3. Results

For each model, we drew 1500 samples from the posterior distributions of parameters for each of 5 independent chains, that after discarding the first 1500 samples of each chain (the burn-in periods). To evaluate MCMC convergence, we use the split $\widehat{R}$ diagnostic, which approaches 1 when chains converge on a single distribution; all of our $\widehat{R}=1\pm 0.03$ [61]. We require high bulk and tail-area effective sample size (ESS) for mortality rates $\mathit{\lambda }$ in order to conduct reliable inference on RIIs; both bulk and tail area ESS were >1900 for all $\mathit{\lambda }$, which is more than sufficient. We also verified that the residuals from the model contain neither SA nor any indication of non-linearity in the relationship between the ICE and log-mortality rates. The computations were completed using parallel processing and the cmdstanr R package [62] on a Dell XPS 13 laptop computer with Intel Core i7-8565U CPU 1.8 GHz, requiring ≈5.25 h per model.

Table 4. Summary of posterior distributions of scalar parameters in the male and female county mortality models.

	Female			Male
	Mean	2.5%	97.5%	Mean	2.5%	97.5%
${\alpha }_1$	−4.863	−4.867	−4.859	−4.375	−4.379	−4.371
${\alpha }_2$	−0.163	−0.236	−0.090	0.005	−0.068	0.077
$\gamma$	0.333	0.218	0.449	0.443	0.341	0.549
$\beta$	−1.661	−1.720	−1.601	−1.823	−1.879	−1.766
${\tau }_{{\varphi }_1}$	0.167	0.153	0.180	0.149	0.137	0.163
${\tau }_{{\varphi }_2}$	0.223	0.058	0.553	0.200	0.028	0.537
${\tau }_{\theta }$	0.059	0.047	0.070	0.067	0.056	0.076
$\mu$	−0.001	−0.042	0.038	−0.001	−0.043	0.039
$\rho$	0.996	0.992	0.999	0.996	0.993	0.999
${\tau }_{\mathcal{X}}$	0.173	0.169	0.177	0.173	0.169	0.177

reports a summary of the posterior distributions of select model parameters. The mean county female mortality rate for ages 55–64 is found to be $e^{-4.863}=773$ per 100,000 [770, 776] for the U.S. mainland. Female mortality in Hawaii is estimated to be $e^{-0.162}=0.85$ [0.79, 0.91] times female mortality in the (48) continental states. For males, the corresponding mean county mortality rate is 1259 per 100,000 [1254, 1264], with no difference in Hawaii. For both male and female mortality, the scale of the spatial components of the BYM model, ${\tau }_{{\varphi }_1}$ and ${\tau }_{{\varphi }_2}$, are substantially larger than the spatially unstructured component ${\tau }_{\theta }$. Thus, net of the ICE, large-scale regional trends account for more variation in county mortality rates than do heterogeneous local characteristics.

Table 5. County mortality rates, ages 55–64, by sex and select quantiles and the corresponding relative indices of inequality (posterior means with 95% CIs).

a. Mortality Per 100,000
	1%	10%	20%	80%	90%	99%
F	391 [382, 400]	529 [523, 536]	602 [596, 607]	1001 [993, 1010]	1116 [1105, 1128]	1408 [1,375, 1,444]
M	627 [612, 641]	865 [855, 874]	980 [972, 989]	1631 [1619, 1644]	1823 [1807, 1840]	2,309 [2261, 2361]
b. Relative Index of Inequality
	$\frac{p80}{p20}$		$\frac{p90}{p10}$		$\frac{p99}{p1}$
F	1.66 [1.64, 1.68]		2.11 [2.08, 2.14]		3.60 [3.48,3.72]
M	1.66 [1.64, 1.68]		2.11 [2.08, 2.14]		3.68 [3.57, 3.81]

reports select quantiles of mortality rates for all counties with corresponding RIIs. When ordered by their estimated mortality rates, both male and female mortality have equivalent relative rates: ${\mathrm{RII}}_5=1.66$ and ${\mathrm{RII}}_{10}=2.11$. At the extreme, however, the ${\mathrm{RII}}_{100}=\frac{p99}{p1}=3.60$ [3.48, 3.72] for females and 3.68 [3.57, 3.81] for males.

Table 6. ICE-Mortality gradient summarized as equivalent relative index of inequality for select quantiles.

	$\frac{\mathit{p}20}{\mathit{p}80}$	$\frac{\mathit{p}10}{\mathit{p}90}$	$\frac{\mathit{p}1}{\mathit{p}99}$
F	1.35 [1.33, 1.36]	1.62 [1.59, 1.65]	2.84 [2.72, 2.97]
M	1.38 [1.37, 1.40]	1.70 [1.66, 1.73]	3.14 [3.01, 3.28]

Note: Samples from the posterior distribution of β were transformed using g(β) = RII = exp(dx × β)⁻¹. The values reported here are the mean and 95% CI for g(β).

reports the ICE-Mortality gradient $\beta$ in terms of the implied RIIs for select quantiles; for female mortality, ${\mathrm{RII}}_5=1.35$ [1.33, 1.36], whereas for males ${\mathrm{RII}}_5=1.38$ [1.37, 1.40]. These estimates are near the low end of our prior probability distribution for $\beta$.

Figure 12 compares the posterior distribution for $\beta$ under the preferred HBM and under the naive model. For male and female mortality, the distribution is shifted towards zero (attenuated) by the naive model. For the male mortality models, the bulk of the posterior distributions do not overlap each other. Figure 13 shows how uncertainty of the ICE values impacts the probability distributions for the county mortality rates, both in terms of their mean values (estimates) and the width of their 95% CIs (posterior uncertainty). Most of the estimates do not substantively differ, but do have wider CIs. For 41 counties, the absolute difference in male mortality estimates between models is greater than 50 per 100,000, with the maximum absolute difference being 168 per 100,000. The maximum difference in uncertainty for male mortality rates between models is 153 per 100,000, although 69 counties have a difference in uncertainty greater than 50 per 100,000. The largest differences in estimated mortality appear, not surprisingly, in the same counties that have the largest ${\widehat{\Delta }}_i$ from the ICE data model. Note that many of the same counties that have large ICE SEs also have suppressed female mortality data; therefore, differences between models of female mortality are slightly less than are the differences between male mortality models.

5. Conclusions

As access to spatial data products increases, researchers need to be aware of the tradeoffs between data granularity and data quality. This paper offers a methodology for both evaluating data quality and modeling observational uncertainty with spatial survey data, following previously published research on HBMs for spatial data [19,20]. The main contributions of this paper are identifying basic inferential challenges that arise from measurement error with spatial data, and integrating spatial HBMs for survey data into a practical workflow for population health research. Our online supplementary material provides the computer code required to implement the proposed model using the Stan programming language [39].

As researchers aim to take full advantage of new geospatial data products for “precision public health” [63], we caution that data for vulnerable and marginalized populations tends to be the least reliable. Similarly, data quality may impose limitations on our ability to undertake ‘complex’ multivariate analyses. The impacts of observational error on multivariate models are unpredictable; with SA, ignoring such errors may be more treacherous. When a conventional geospatial analysis produces anomalous or implausible results, e.g., that a higher rate of health insurance coverage increases mortality rates in southern Florida [64], researchers and reviewers ought to ask if observational error could be the cause of the findings. Our demonstration analysis shows that ignoring observational uncertainty in a single covariate measured with a fair degree of precision can impact coefficient estimates, model predictions, and posterior uncertainty of estimates. The honest and complete reporting of uncertainty is a critical component of the scientific process and research integrity. Currently accepted practices for analyzing small-area data fall short of this standard, and the widespread adoption of workflows and modeling strategies that incorporate observational uncertainty is called for.

Our analysis of mortality rates benefits from important computational advances of recent years, namely in the application of dynamic Hamiltonian Monte Carlo algorithms to MCMC sampling and Bayesian inference [39,65,66,67]. Nonetheless, computational limitations remain. ACS estimates of population at risk are also subject to sampling error, and it is concerning that researchers employ small-area estimates for highly specific demographic subgroups without considering data quality. Unfortunately, incorporating data models for population at risk estimates into models for count outcomes appears to be a computational bottleneck. The computational limitations of the CAR model also influenced our decision to employ the BYM model specification. Future research may address these challenges. Furthermore, our methodology does not consider the possibility of non-sampling errors in survey estimates. It is possible that errors themselves are correlated due to bias in the sampling design or survey implementation. Modeling potential biases in survey estimates would require additional information.

We recommend that researchers consider data quality to be a core criterion for variable selection, and integral to study design. Analyses of data quality should appear in research protocols, and should be included in the peer review process. The workflow we introduce here is intended to provide a basis for more intensive evaluation and criticism of model-based inferences with small-area data, and to produce models that maintain greater fidelity to researchers’ state of knowledge. When policy and funding decisions are at stake, closer evaluation of priority areas and areas with questionable data quality should be undertaken.