A New Benford Test for Clustered Data with Applications to American Elections

Abstract

A frequent problem with classic first digit applications of Benford’s law is the law’s inapplicability to clustered data, which becomes especially problematic for analyzing election data. This study offers a novel adaptation of Benford’s law by performing a first digit analysis after converting vote counts from election data to base 3 (referred to throughout the paper as 1-BL 3), spreading out the data and thus rendering the law significantly more useful. We test the efficacy of our approach on synthetic election data using discrete Weibull modeling, finding in many cases that election data often conforms to 1-BL 3. Lastly, we apply 1-BL 3 analysis to selected states from the 2004 US Presidential election to detect potential statistical anomalies.

1. Introduction

Although Benford’s Law (see Section 2 for a review) has successfully detected malfeasance in many settings (flagging, among others, financial records, images, and scientific data that have been modified), the difficulty in determining when one should expect Benfordness severely limits its applications to detect malfeasance or other issues. One popular argument is that data sets that are spread out over several orders of magnitude are often close to Benford. Though this phenomenon is frequently true (see [1,2] for results on exponential random variables), it can fail; see for example Chapter 2 of [3] or [4] for issues with the ‘spread’ hypothesis.

Clearly, not all data sets follow Benford’s law, especially in elections [5,6]. For example, if the data is not spread out sufficiently, the distribution of first digits is unlikely to conform to standard base 10 representations of Benford’s law (hereafter referred to as 1-BL 10); thus, deviations from Benfordness do not suggest malfeasance or other concerns. This issue is the main motivation for this work; specifically, our analysis is intended to address discussions where claims of election malfeasance are made when observed candidate vote counts do not satisfy 1-BL 10 [7].

A classic example is precinct by precinct election data. Figure 1 provides such data from three counties from the 2004 US Presidential Election between incumbent President George W. Bush and Senator John F. Kerry. As is apparent in Figure 1, precinct level election data can be clustered thus rendering 1-BL 10 useless. As a result, work by Walter Mebane and others have looked at the second digit (hereafter referred to as 2-BL 10), or the last two digits (which one expects to be the same about 10% of the time base 10). For example, Mebane (2006) applied 2-BL 10 to election data from the state of Florida in 2004 US Presidential Election and the 2006 Mexican Presidential Election [8]. In subsequent research, Mebane (2009) applied 2-BL 10 to the 2009 Presidential Election in Iran to detect potential instances of malfeasance [9]. Mebane (2012) applied these techniques to German and American elections, discussing manifestations of coercion as well as effects of strategy and gerrymandering [10]. Although there have been some criticisms of the usefulness of Benford’s law [5], Mebane (2011) responded to these criticisms arguing that 2-BL 10 does indeed have a reasonable degree of potential usefuless depending on the nature of the data analyzed and associated levels of aggregation [6]. There are many other applications of Benford’s law to election data as well [11,12,13,14,15,16,17].

In this study, we propose a novel approach for digit-based analysis of concentrated data. Namely, we write the data in terms of base 3 instead of base 10; the advantage of doing so is that our numbers are now spread out over several more magnitudes with two choices for the first digit instead of nine. For example, 81 is a two digit number in base 10, but in base 3 it is ${\mathrm{10,000}}_3$, five digits representing four orders of magnitude. The transformation of vote counts to base 3 thus spreads out clumped data. This idea of switching bases was originally proposed by the fourth named author over a decade ago to detect falsified data that passes a Benford test base 10 but might fail it in another base. Other base representations are of course possible; however, higher base representations would cluster the data more than base three, and base two representations would only have a single choice of a leading digit. Base 3 representations in election data were recently applied by the Kossovsky and Miller (2020) in [18].

This paper thus contributes to the literature on Benford’s law by performing a first digit analysis of precinct level election data based on base 3 representations (hereafter referred to as 1-BL 3). Specifically, our work is organized as follows. First, we discuss existing forms of Benford’s law used for analyzing data and various generalizations. Subsequently, we fit precinct-level election data from the 2004 US Presidential Election to discrete Weibull distributions and assess the goodness of fit of these distributions in modeling election data. Afterwards, we draw upon this analysis by using a variety of choices of parameters based on this estimation to generate synthetic election data to compare the performance of 1-BL 10 and 1-BL 3. We find that, in many cases, election data generally conforms to 1-BL 3, but does not to 1-BL 10. Lastly, we apply 1-BL 3 analysis to selected states from the 2004 US Presidential election to detect potential statistical anomalies. We emphasize that our work is generalizable to other elections and thus should be a useful tool for researchers to use in analyzing the integrity of elections.

2. Benford’s Law and Generalizations

In the 1880s Simon Newcomb [19] observed that the pages of tables of logarithms were not equally worn; there was more wear and tear on the earlier pages (those corresponding to the logarithms of numbers with a small first significant digit, such as 1, 2 or 3) than the later pages (those from numbers with a large first digit (such as 7, 8 or 9). Fifty years later, Frank Benford saw similar biases in many different data sets including population, X-ray volts, and the area of rivers [20]. Today, it has been found applicable in a broad number of disciplines such as geology, accounting, economics, computer science, and others [21,22,23,24,25,26].

Any positive number x can be written base B as $x=S_B\left(x\right)·B^{k\left(x\right)}$, where $S_B\left(x\right)\in [1,B)$ is the significand and $k\left(x\right)$ is an integer. We now say a data set is Benford, or satisfies Benford’s Law, base B if

$$\mathrm{Prob}\left(\mathrm{First}\mathrm{digit}\mathrm{base}B\mathrm{is}d\right)={log}_B\left(1+\frac{1}{d}\right);$$

(1)

thus base 10 the probabilities decrease from roughly 30% for a leading digit of 1 down to about 4.6% for a 9.

Sometimes, however, one uses Benford to refer to not just the first digit, but the distribution of the significand (alternatively one often says the system is strongly Benford); thus

$$\mathrm{Prob}\left(\mathrm{Significand}\mathrm{base}B\mathrm{is}\mathrm{at}\mathrm{most}s\right)={log}_B\left(s\right).$$

(2)

Note that the first digit probabilities follow immediately from the significand, as

$$\begin{array}{cc}\hfill \mathrm{Prob}\left(\mathrm{First}\mathrm{digit}\mathrm{base}B\mathrm{is}d\right)& =\mathrm{Prob}\left(\mathrm{Significand}\mathrm{base}B\mathrm{is}\mathrm{in}[d,d+1)\right)\hfill \\ \hfill & ={log}_B\left(d+1\right)-{log}_B\left(d\right)={log}_B\left(1+\frac{1}{d}\right),\hfill \end{array}$$

(3)

and we can easily determine the probabilities of the second digits:

$$\mathrm{Prob}\left(\mathrm{Sec}\mathrm{ond}\mathrm{digit}\mathrm{base}B\mathrm{is}d\right)=\sum _{k=1}^{B-1}{log}_B\left(1+\frac{1}{Bk+d}\right).$$

(4)

We see a set is Benford base B if and only if its logarithms base B are equidistributed modulo 1; thus the digit bias disappears under a logarithmic transformation. For more on the history, theory and applications of Benford’s law see the survey articles [27,28,29,30] and the books [3,31,32].

Table 1. Benford probabilities for first and second digit (to four decimal places).

d	Probability First Digit d	Probability Second Digit d
0		0.1197
1	0.3010	0.1139
2	0.1761	0.1088
3	0.1249	0.1043
4	0.0969	0.1003
5	0.0792	0.0967
6	0.0669	0.0934
7	0.0580	0.0904
8	0.0512	0.0876
9	0.0458	0.0850

provides 1-BL 10 and 2-BL 10 probabilities, namely the probability of a first or second digit being d base 10. Figure 2 provides a visual distribution of the first digits.

Table 2. Benford probabilities for first digit base 3 (to four decimal places).

d	Probability First Digit d	Probability Second Digit d
0		0.4022
1	0.6309	0.3247
2	0.3691	0.2732

provides analogous probabilities for the distribution of the first and second digits of 1-BL 3 and second digit base 3 (2-BL 3) probabilities, and Figure 3 provides a visual representation of the distribution of the leading digits under base 3 representations.

One of the most utilized tests to see if a set is Benford is to compute the chi-squared statistic, with comparisons of the associated cumulative distribution function to statistical significance levels, usually $\alpha =0.01$ or 0.05. If $E\left(d\right)$ is the expected number that have a first digit of d base B, and $O\left(d\right)$ is the observed number, then the chi-squared statistic is

$${\chi }^2=\sum _{d=1}^{B-1}\frac{{(O\left(d\right)-E\left(d\right))}^2}{E\left(d\right)}.$$

(5)

If we work in base $B=10$ then as we have 9 choices for the first digit and the probabilities sum to 1, we only have 8 degrees of freedom (the probability of the last digit is forced from the other 8, and zero is never a first digit). If the digit distribution is drawn from Benford’s law, then $95\%$ of the time we should have a chi-square value of $15.5$ or less, and $99\%$ of the time it should be $20.1$ or less. To put it another way, if the data follows Benford’s law and we observe a value greater than $20.1$, there is only a $1\%$ chance of that happening given that the data is Benford; if you observe values larger than $20.1$ the probability falls rapidly.

If we work in base 3, there are only two possibilities for the leading digits. For a set that is Benford we expect to observe a leading digit of 1 approximately 63.093% of the time and of 2 about 36.907%, with the chi-square values (with one degree of freedom) being 3.84 (at the 95% level) and 6.63 (at the 99% level).

In the following section, we modeled the skewed distributions of precinct level election data for the purposes of analyzing these distributions with respect to Benford’s law.

3. Empirical Estimates Using 2004 Election Data

3.1. Modeling with the Discrete Weibull

As is apparent in Figure 1, precinct by precinct election data can be skewed by vote counts heavily occurring in a just a few precincts. The discrete Weibull distribution provides a flexible family of distributions useful for modeling this type of skewed data [33]. The family, as well as associated variants, have been used to model real-world phenomena including soccer, respiratory capacity, fertility, and levels of microorganisms among others [34,35,36,37,38]. In this section, we utilize this family to model precinct by precinct election data.

Specifically, in state i and county j consisting of $k=\{1,\dots ,K\}$ precincts, we can model the vote count for two candidates x and y, $x_{i,j}=\{x_{i,j,1},\dots ,x_{i,j,K}\}$ and $y_{i,j}=\{y_{i,j,1},\dots ,y_{i,j,K}\}$, using a discrete Weibull representation [33]. We take

$$\begin{array}{c}\hfill P(x_{i,j,k};{\alpha }_{x,i,j},{\beta }_{x,i,j})=\exp \left[-{\left(\frac{x_{i,j,k}}{{\alpha }_{x,i,j}}\right)}^{{\beta }_{x,i,j}}\right]-\exp \left[-{\left(\frac{x_{i,j,k}+1}{{\alpha }_{x,i,j}}\right)}^{{\beta }_{x,i,j}}\right]\end{array}$$

(6)

and

$$\begin{array}{c}\hfill P(y_{i,j,k};{\alpha }_{y,i,j},{\beta }_{y,i,j})=\exp \left[-{\left(\frac{y_{i,j,k}}{{\alpha }_{y,i,j}}\right)}^{{\beta }_{y,i,j}}\right]-\exp \left[-{\left(\frac{y_{i,j,k}+1}{{\alpha }_{y,i,j}}\right)}^{{\beta }_{y,i,j}}\right],\end{array}$$

(7)

where $\alpha$ is the scale parameter and $\beta$ is the shape parameter.

3.2. Data and Model Estimation

We estimated the discrete Weibull model on data from the 2004 Presidential Election between incumbent President George W. Bush and Senator John F. Kerry available online on Harvard University’s DataVerse repository [39]. The repository is a comprehensive database providing precinct by precinct vote counts across counties for many states of the 2004 as well as other Presidential, Senatorial, and Congressional elections. Although there were a number of other candidates appearing on the ballot in the general election, we focused our analysis on the two main candidates in several selected states. Specifically, to estimate the model on states with varying political inclinations, we analyzed one “red” state (North Carolina), one “blue” state (Vermont), and three “battleground" states (Colorado, Ohio, and Wisconsin). We verified the data correctly represents vote counts reported by the United States Federal Election Commission for all five states, with the exception of Wisconsin, having an error of less than $0.6\%$ for each candidate [40]. In order to ensure a reasonable sample size for our analysis, we restricted our analysis to counties consisting of more than ten election day precincts.

For such counties, we estimated the scale and shape parameters for the distribution of the precinct by precinct vote tallies for each of the two candidates x and y in state i and county j—${\alpha }_{x,i,j}$, ${\beta }_{x,i,j}$, ${\alpha }_{y,i,j}$, and ${\beta }_{y,i,j}$—via the method of maximum likelihood (MLE). Subsequently, to assess goodness of fit, we performed a series of two sample Kolmogorov-Smirnov (KS) tests assessing the null hypothesis of distributional equality, comparing the distribution of $x_{i,j}$, or $y_{i,j}$, with a random sample of size K of the associated discrete Weibull distribution using parameter estimates obtained via MLE. We averaged p-values obtained from 100 simulations to obtain a pooled Monte Carlo estimate of the goodness of fit of the estimated distributions. Our estimated shape and scale parameters, as well as pooled p-value estimates, for the battleground states of Ohio, Colorado, and Wisconsin are presented in the scatterplots in Figure 4 for counties having at least ten precincts. Scatterplots of MLE estimates and KS-test results for Vermont and North Carolina are omitted for brevity and are available from the authors upon request.

As Figure 4 illustrates, for all three battleground states, the discrete Weibull distributions fit quite well. Altogether there are three counties in Ohio that have pooled p-values less than 0.05, one county in Wisconsin, and none in Colorado for Bush. For Kerry, there were eight such counties in Ohio, one county in Wisconsin, and none in Colorado. After performing the standard, albeit conservative, Bonferroni corrections there was only one county—Cuyahaga County, Ohio—for Bush, that was statistically significant ($p<4\times {10}^{-5}$). These results demonstrate the efficacy of using discrete Weibull distribution to model precinct level data across counties.

Furthermore, as Figure 4 also illustrates, our estimated parameters vary notably, depending on the county.

Table 3. Maximum Likelihood Estimates and Goodness of Fit Test Results for Vote Counts for Selected Counties in 2004 US Presidential Election—President George W. Bush. p-values above 0.05 (plotted in blue) indicate conformance to discrete Weibull distribution.

	${\widehat{\mathit{\alpha }}}_{\mathit{Bush},\mathit{i},\mathit{j}}$	${\widehat{\mathit{\beta }}}_{\mathit{Bush},\mathit{i},\mathit{j}}$	p-Value	Number of Precincts
Milwaukee County, WI	351.457	1.410	0.321	560
Stark County, OH	287.717	2.289	0.472	364
Grand County, CO	205.389	2.706	0.726	12

and

Table 4. Maximum Likelihood Estimates and Goodness of Fit Test Results for Vote Counts for Selected Counties in 2004 US Presidential Election—Senator John F. Kerry. p-values above 0.05 indicate conformance to discrete Weibull distribution.

	${\widehat{\mathit{\alpha }}}_{\mathit{Kerry},\mathit{i},\mathit{j}}$	${\widehat{\mathit{\beta }}}_{\mathit{Kerry},\mathit{i},\mathit{j}}$	p-Value	Number of Precincts
Milwaukee County, WI	598.1	2.2	0.223	560
Stark County, OH	289.057	4.085	0.304	364
Grand County, CO	167.35	3.188	0.614	12

provide MLE parameter estimates and p-values the KS tests, and precinct sizes for the selected counties presented in Figure 1.

Table 3 and Table 4 illustrate the robustness of the discrete Weibull distribution in its ability to model different types of counties across the country. Namely, Milwaukee county, WI has larger scale and smaller shape parameters than the less densely populated Stark County, OH, and Grand County, CO. Regardless, the p-values for all three counties are well above 0.05, demonstrating the robustness of the choice of the discrete Weibull distribution across different counties.

The analysis in this section illustrates the efficacy of using the discrete Weibull distribution at fitting precinct level election data across a wide choice of parameterizations. In the following section, we utilize these facts to examine the usefulness of 1-BL 3 and 1-BL 10 in analyzing election data.

4. Comparisons of 1-BL 3 and 1-BL 10 to Potential Vote Distributions

In the previous section, we demonstrated the usefulness of the discrete Weibull distribution in modeling precinct level election data. In this section, we examine a variety of choices of discrete Weibull parmaeterizations and assess their adherence to 1-BL 3 and 1-BL 10 via Monte Carlo analysis using both chi-squared testing and binomial testing.

4.1. Simulations Using Chi-Squared Testing

Specifically, based on the results presented in Figure 4, we simulated county level data with the number of simulated precincts varying between 50 and 300. We varied the scale parameter ${\alpha }_{x,i,j}$ between 100 and 600 in increments of 25 and varied the shape parameter ${\beta }_{x,i,j}$ between 1 and 6 in increments of 1. After compiling the expected and observed number of first digits under both base 10 and base 3 representations, we performed chi-squared tests for both 1-BL 10 and 1-BL 3 (assuming under the null hypothesis of distributional equality, chi-squared distributions having eight and one degree of freedom respectively as discussed in Section 2) and subsequently computed associated p-values. We averaged these p-values over the course of 25 Monte Carlo simulations. Our results are presented in Figure 5 and Figure 6.

As is apparent Figure 5, assuming a statistical significance level of $\alpha$ = 0.05 the simulated precinct data rarely conforms to 1-BL 10 except for choices of scale parameter ${\alpha }_{x,i,j}$ between 100 and 200. Upon conversion to base 3, adherence to Benford’s law is considerably more robust, as is illustrated in Figure 6. Even under base 3 representations, however, Benford’s law is not always observed. For example, if the scale parameter ${\alpha }_{x,ij}$ exceeds 500 and the shape parameter ${\beta }_{x,i,j}$ exceeds 4, the associated p-values are below 0.05 and thus Benford’s behavior is not observed. (This behavior persists even after altering $\alpha =0.05$ for multiple comparisons via Bonferroni corrections to $\alpha =1.67\times {10}^{-4}$.)

4.2. Simulations Using Binomial Testing

Although chi-squared testing has been used extensively in Benford’s law analysis of elections [6,8,9,10,11,12,13,14,15,16,17], some have raised concerns with the use of the chi-squared test in certain situations [41]. (For example, although it depends on the application, there were nearly half a million observations of some hydrology data in [21], the chi-square test was sensitive enough to detect as significant very small deviations from Benford’s law, something that would not happen with essentially pure geometric behavior, such as the Fibonacci numbers, which do pass a chi-square test with this many values). As there are only two possible leading digits, one alternative approach is binomial testing. We performed the same simulations presented in the prior section instead using two-sided binomial testing to test the null hypothesis of adherence to expected frequencies under 1-BL 3. Our results using the same choices of parameterizations used in Figure 5 and Figure 6 are presented in Figure 7.

As Figure 7 illustrates, the binomial test indicates that 1-BL 3 is indeed useful for modeling election data under a variety of paramaterizations. Altogether, our analysis of this synthetic election data using the wide choices of parameters motivated by our MLE estimation in Section 3 illustrates that 1-BL 3 is considerably more effective than 1-BL 10 in fitting election data. Thus, base 3 representations of precinct by precinct data are considerably much more likely to adhere to Benford’s law than standard base 10 representations for most of the choices of parameterizations presented in Figure 5 and Figure 6. For larger sample sizes, however, the chi squared test in some cases does not detect conformity to 1-BL 3, which may be due to the sensitivity of the test itself as well as slight approximation error [2,21]. Regardless, these results illustrate that for a variety of parameterizations and sample sizes, simulated discrete Weibull data adheres to 1-BL3. As our analysis in Section 3 demonstrates that the American election data analyzed in this study fits these discrete Weibull distributions, our simulations in this section thus provide theoretical justification for applying 1-BL 3 to American election data, which we perform in the following section.

5. Applications of 1-BL 3 to the 2004 US Presidential Election

As a result of our analysis in Section 4, we now have a theoretical justification for applying 1-BL 3 to election data. In this section, we do so by utilizing the data from the 2004 US Presidential Election discussed in Section 3. Specifically, we analyzed counties amongst the states of Colorado, North Carolina, Ohio, Vermont, and Wisconsin to assess adherence to 1-BL 3 on a county by county basis. The electoral outcomes of these states is summarized in

Table 5. States from 2004 US Presidential Election used for Analysis.

State	George W. Bush (R)	John F. Kerry (D)	Number of Counties
North Carolina	56.02% (1,961,166)	43.58% (1,525,849)	100
Vermont	38.80% (121,180)	58.94% (184,067)	16
Wisconsin	49.32% (1,478,120)	49.70% (1,489,504)	72
Ohio	50.81% (2,859,768)	48.71% (2,741,167)	88
Colorado	51.69% (1,101,255)	47.02% (1,001,732)	64

.

5.1. Chi-Square Test Results

As we did in Section 3, we restricted our analysis to counties voting on election day that consisted of at least ten election day precincts. We assumed a statistical significance level $\alpha =0.05$ and performed comparisons of observed and expected values. Specifically, as discussed in Section 2, we did so by utilizing a chi-squared distribution with one degree of freedom to test the null hypothesis of distributional equality between these observed and expected counts. Afterwards, we computed adjusted p-values using the Benjamni-Hochberg procedure to control for the false discovery rate [42] as has been done in other election related work using Benford’s law (see for example [8,43,44]). Our results are presented in

Table 6. First Digit Base 3 Benford’s Analysis on Selected Battleground and Non-Battleground States in the US 2004 Presidential Election.

State	County	Candidate	${\mathit{\chi }}^2$ Stat	p-Value	Number of Precincts	Adjusted p-Value
Colorado	Arapahoe	John F. Kerry	21.152	<0.001	364	0.001
	El Paso	John F. Kerry	20.513	<0.001	378	0.001
	Jefferson	George W. Bush	38.050	<0.001	324	<0.001
	Jefferson	John F. Kerry	108.747	<0.001	324	<0.001
North Carolina	No Anomalies Detected
Wisconsin	No Anomalies Detected
Ohio	Ashtabula	John F. Kerry	39.385	<0.001	127	<0.001
	Butler	John F. Kerry	14.594	0.001	289	0.013
	Geauga	George W. Bush	15.196	<0.001	96	0.013
	Geauga	John F. Kerry	20.812	<0.001	96	0.001
	Greene	George W. Bush	13.860	0.001	142	0.016
	Greene	John F. Kerry	36.189	<0.001	142	<0.001
	Lorain	John F. Kerry	43.509	<0.001	239	<0.001
	Miami	John F. Kerry	14.669	0.001	82	0.013
	Muskingum	John F. Kerry	13.971	0.001	85	0.016
	Portage	John F. Kerry	27.249	<0.001	129	<0.001
	Summit	John F. Kerry	68.917	<0.001	475	<0.001
Vermont	No Anomalies Detected

.

As Table 6 illustrates, all counties amongst the non-competitive states of North Carolina and Vermont as well as the battleground state of Wisconsin conform to 1-BL3. On the other hand, there are four distributions amongst counties/candidates in Colorado and fifteen distributions in Ohio that are statistically anomalous.

However, as Figure 6 from Section 4 demonstrates, depending on the distribution of votes for either candidate, certain counties may not necessarily conform to 1-BL 3 to begin with. As a result, we performed another series of Monte Carlo simulations for the anomalous counties in Table 6 to assess whether the estimated vote distribution for each county/candidate indeed conforms to Benford’s law. To do so, we again ran 25 Monte Carlo simulations drawing from discrete Weibull distributions using the county’s estimated parameters (presented as part of Figure 4) and reported precinct sizes for the anomalous counties in Table 6 to assess whether these distributions indeed conform to 1-BL 3. We averaged p-values across the 25 simulations and compared these values to those in Table 6. We flagged counties where the estimated distribution conforms to Benford’s law (p-value > 0.05) while the actual data does not (adjusted p-value <0.05). Our results are presented in

Table 7. Analysis of Whether Anomalous Counties from Table 6 Deviate from Benford’s Law.

County	Candidate	${\mathit{\alpha }}_{\mathit{x},\mathit{i},\mathit{t}}$	${\mathit{\beta }}_{\mathit{x},\mathit{i},\mathit{t}}$	Number of Precincts	p-Value (KS-Test)	Should Conform to BL but Fails?
Arapahoe	John F. Kerry	326.634	2.973	364	0.003
El Paso	John F. Kerry	231.420	2.499	378	0.507	*
Jefferson	George W. Bush	400.330	3.650	324	0.001
Jefferson	John F. Kerry	352.800	4.717	324	<0.001
Ashtabula	John F. Kerry	208.527	4.538	127	<0.001
Butler	John F. Kerry	218.783	2.869	289	0.037
Geauga	George W. Bush	348.690	4.200	96	0.003
Geauga	John F. Kerry	228.818	3.869	96	0.099	*
Greene	George W. Bush	383.344	2.628	142	0.395	*
Greene	John F. Kerry	244.027	2.272	142	0.494	*
Lorain	John F. Kerry	367.292	2.964	239	0.074	*
Miami	John F. Kerry	235.479	4.963	82	0.010
Muskingum	John F. Kerry	215.593	3.613	85	0.054	*
Portage	John F. Kerry	347.487	4.109	129	<0.001
Summit	John F. Kerry	365.481	3.712	475	<0.001

* The p-values presented in Table 7 are estimates before correcting for multiple comparisons; therefore, our approach can be considered conservative as it provides a lower bound on the number of statistically anomalous counties, and thus likely understates the number of anomalous counties.

.

As Table 7 shows, the vote distribution for Senator John F. Kerry in El Paso county, Colorado should theoretically adhere to 1-BL 3 but does not. There were four additional counties in Ohio for Kerry that should also have conformed to 1-BL 3 that failed (Geuga, Greene, Lorain, and, Muskingum Counties). There was one such county for President George W. Bush—Greene County, Ohio. Furthermore, as Table 7 also illustrates, although several other distributions for both candidates were deemed statistically anomalous in Table 6, their estimated distributions do not either, and thus their non-adherenece to 1-BL 3 may not be unnatural.

5.2. Binomial Test Results

For comparative purposes, we utilize the binomial testing approach analyzed in Section 4.2 in this subsection. For brevity, we present results using these tests for two counties from the state of Colorado, one anomalous and one not, and further analysis of other counties and other states is available from the authors upon request; the analysis shows the tests agreeing on when fits are Benford and when they are not. Using chi-squared testing, Jefferson County, CO has been determined to be statistically anomalous for both candidates (p-value < 0.01). Specifically, for George W. Bush, 1 appears as the first digit 258 times across precincts, whereas 2 appears as the first digit 66 times under base 3 representation. For John F. Kerry, on the other hand, 1 appears as the first digit 295 times and 2 appears as the first digit 29 times. The expected counts for the first digits being 1 or 2 are 204.42 and 119.58 respectively. For Bush and Kerry, the 95% confidence intervals are (242.44, 271.77) and (283.17, 304.31) respectively for 1 being the first digit and (52.23, 81.56) and (19.69, 40.83) for 2 being the second digit. As a result, the digits of neither candidate belong to the associated CIs, and thus for both candidates, the county is statistically anomalous confirming the chi squared test results.

Denver County, CO, however, is not anomalous according to chi-squared testing (p-values 0.48 and 0.79 respectively). For George W. Bush, 1 appears as the first digit 253 times across precincts, whereas 2 appears as the first digit 162 times. For John F. Kerry, on the other hand, 1 appears as the first digit 253 times and 2 appears as the first digit 167 times under base 3 representation. The expected counts for the first digits being 1 or 2 are 264.99 and 155.01 respectively. For the two candidates, the 95% confidence intervals are (239.67, 277.65) and (232.59, 272.69) respectively for 1 being the first digit and (142.35, 182.33) and (147.21, 187.40) respectively for 2 being the second digit. In this instance, both digits for the two candidates belong to their respective CIs, thus confirming the results indicated by the chi squared test.

6. Discussion

The results in Table 7 flag several counties for Senator Kerry and one for President Bush as statistically anomalous in terms of 1-BL 3. Interestingly, Ohio was one of the most contentious states in the 2004 election [45]. Although our model flags approximately five percent of Ohio’s 88 counties, these statistical anomalies are still detected after corrections due to multiple comparisons. These results, however, neither offer any indication as the cause of potential malfeasance, if present, nor do they shed light on which side was responsible. One goal of Benford tests is to detect malfeasance; however, certain types cannot be observed by these methods. For example, if a fixed percentage of votes were switched from one candidate to another while keeping the number of voters fixed, the net effect is to just change the percentage of support for each candidate. Similarly, if a fixed number of ballots are added in each precinct, the effect would be to both increase the number of voters and adjust the candidates’ support. It is important to remember tests such as these cannot prove malfeasance; the main utility is to highlight situations that are worth additional investigation; for example, in cases where 1-BL 3 fails it might be valuable to check current precinct voting against previous years when 1-BL 3 held.

Again, although malfeasance could constitute one explanation of non-conformance to Benford’s law, associated anomalies do not necessarily prove malfeasance as other explanations are be possible. For example, Mebane (2010, 2012, 2013) has discussed how deviations from 2-BL 10 could be due not only to malfeasance but also may be manifestations of mobilization efforts, last minute changes to voting preference, as well as effects of gerrymandering among others [10,12,14]. The effect of these factors on conformance 1-BL3 has not been studied thus far and is a worthy topic of future research. If 1-BL3 can indeed manifest these effects, it would be useful for future research to differentiate between these effects and malfeasance, to render these techniques more useful for auditing or potential litigation.

7. Conclusions and Future Research

Our study illustrates the usefulness of 1-BL 3 in analyzing elections, flagging a number of counties in the 2004 US Presidential Election. Our analysis in Section 4, suggests that 1-BL 10 analysis would almost surely unnecessarily flag many other counties that may not truly be anomalous. As discussed in Section 6, however, although potential malfeasance could constitute one explanation of 1-BL 3 based anomalies, there may be other potential explanations as well. Future research should look into refining these number theoretic models to differentiate amongst potential explanations. Regardless, if there are concerns about potential malfeasance, 1-BL 3 could indeed be a useful tool to detect for anomalies as a basis for potential auditing.

Although we have analyzed several states from the 2004 US Presidential Election, we hope our work encourages future research to apply our method to other elections of all levels, both American and foreign. However, as noted by Mebane (2011), the appropriate unit of analysis for Benford’s law is precinct level data across counties, as we have utilized in this study [6]. As a result, Benford’s law is not useful for detecting potential malfeasance with absentee ballots such as in the 2018 Congressional election in North Carolina’s ninth district, but there are other statistical techniques that have been demonstrated to be useful for such absentee ballot analysis [46]. The number of statistical tools to analyze the integrity of elections is growing [43,47,48,49,50], and the techniques presented in this paper thus should be viewed as yet another tool in the statistician’s arsenal to analyze election data. Future research should thus include our tool as part of a suite of such tools to comprehensively analyze all aspects of election data.

One potential limitation of this approach is that the data may become so heavily spread out via base three representations that some instances of malfeasance may be rendered undetectable. Other base representations may potentially ameliorate this problem at the expense of clustering the data. Future research should weigh such tradeoffs among different bases.

Moreover, until now, 2-BL 10 was the primary technique used for applying Benford’s law to election data. Future research should compare the efficacy of 1-BL 3 to 2-BL 10. It would be useful to simulate election data as we have done in the study, externally impose malfeasance in the model, and assess what forms of malfeasance are detectable by these techniques. Finally, there is of course no a priori reason to apply these techniques solely to elections, and we hope that this research encourages the use of 1-BL 3 in other applied settings of clustered data where 1-BL 10 is not applicable. Batting averages in baseball, serving speeds in tennis, government-reported statistics, and stock market data are indeed worthy examples of future investigation.

It is always useful to have tools to identify potential anomalies in election data. We hope the techniques presented here provide a useful addition to the applied mathematician/statistician’s toolbox for doing so.