The Analysis and Deinterleaving of Periodic Point Processes

Abstract

This paper addresses the problem of determining the underlying structure of a given point process. The point process data is a finite set of event times, given by a set of real numbers. We wish to determine if the process has been generated by one or more periodic processes and, if so, extract the period or periods from the data. If there are several periods, we also wish to deinterleave the data into separate periodic processes, each generated by a single period. We approach the problem by developing two algorithms. These algorithms are designed to work on all data sets, but, in particular, on extremely sparse data sets where other procedures do not work. The first algorithm works on sets of event times with only one underlying period, quickly producing an estimate of that period. The mathematical justification of the procedure involves number theory, including an interpretation of the Riemann zeta function as an asymptotic probability distribution. The second algorithm analyzes event times from multiple periodic processes. It relies on the first algorithm as an “engine” to create larger data sets, from which it produces estimates of underlying periods. This second procedure also relies on a mathematical justification, a result from both number theory and harmonic analysis—the equidistribution theorem of Weyl. We then deinterleave the data, breaking it down into components, each generated by a single period.

1. Introduction

Point processes are an important component of data analysis, appearing in everything from analysis of astronomy data, atmospheric chemistry, radar and sonar, to the analysis of neural networks. This paper deals with analyzing a given point process, and in particular, determining if the process has been generated by one or more periodic processes. The point process data is a finite set of event times, given as a set of real numbers. The goal is to determine if the data has been generated by periodic processes, and if so, extract the period or periods from the data. If there are several periods, we then deinterleave the data into separate periodic processes, each generated by a single period.

Our approach is to create two algorithms to analyze the data. The algorithms rely on the structure of randomness (see Section 2 and Section 3), which tells us that random data can settle into a structure based on the set from which the data is extracted. This approach to solving the data analysis problems allows us to bring in some deep mathematical ideas, which in turn explain why the algorithms work so efficiently. The algorithms are designed to work on all event-time data, including data where other methods break down because of sparsity (see Section 2.4 for comparisons with least squares and periodograms). The first algorithm works on event times with only one underlying period, efficiently producing an estimate of that period. The second works on sets of event times with several underlying periods, producing estimates of those periods, which in turn leads to deinterleaving the data into components, each generated by a single underlying period. We present the algorithms and provide mathematical justifications as to why they work.

The first procedure is the modified Euclidean algorithm or MEA. (The steps in the algorithms and their mathematical justifications led naturally to their names.) The algorithm is extremely efficient, stable given reasonable noise, and converges very quickly [1,2,3,4]. Numerous applications of the MEA are discussed in [1,2,3,4,5].

The second procedure, the equidistributed MEA or EQUIMEA, works on event-time data, and relies on the MEA as an “engine” to create larger data sets, from which it produces estimates of the underlying periods. We then deinterleave the data, breaking it down into components, each generated by a single period. Several applications of the EQUIMEA are discussed in [6].

The MEA and EQUIMEA work on all reasonable event-time data. The assumption is that data elements are the results of periodic processes, e.g., radar or sonar data, with missing observations and additive noise. The noise is jitter noise, a variation around the exact time of a given event. The missing data elements can be visualized by seeing the event times as zero crossings of sinusoids, where a random process has removed zero crossings. With no missing observations, standard methods, e.g., least squares or periodograms, can extract the periods even in a relatively noisy environment. If, however, many of these zero crossings are missing, these methods will break down (see Section 2.4).

The EQUIMEA and deinterleaving problem is quite important, especially in radar. Histogram methods for deinterleaving radar can be found in [7,8]. The paper [9] features histograms and neural networks, while histograms and AI are featured in the recent article [10]. Clustering methods can be found in [11]. The Lomb–Scargle periodogram has been used in astronomy [12], atmospheric chemistry [13], and biology [14].

The MEA has been compared several times with standard methods, and these references can be found [1,2,3,4].

The mathematical justification of the MEA involves number theory, including an interpretation of the Riemann zeta function as an asymptotic probability distribution. This interpretation allows us to prove Theorem 1, which shows that in an arbitrarily large lattice of positive integers in n-dimensional Euclidean space, if we randomly choose an element of the lattice, the probability that the n-tuple is relatively prime quickly converges to 1 as n increases. Given noise-free data, the algorithm very likely converges to the exact value of the period with as few as ten data samples.

Remark 1. 

The mathematical reasons that only ten data elements are needed come from Theorem 1, estimates given in Proposition 4, and calculations of the zeta function given in Table 1. Computations from computer simulations in Table 2 demonstrate the precision of this result.

For data with additive noise, simulation results show, for example, good estimation of the period, converging as noise decreased and the number of data points increased (see Table 3 for computer simulations of these results). Variations on the MEA, which deal with very noisy environments, can be found in [2,3,4].

The EQUIMEA procedure works on periodic processes created by several sources, each with different underlying periods. We assume that the periods are independent. Therefore, with probability one, they are not rational multiples of each other. We note, however, that even if two or more periods are rational multiples, the algorithm can still produce results by utilizing the phase data. The general algorithm extracts the set of fundamental periods by using Weyl’s equidistribution theorem to help separate sources. Given two periods, differences of their respective period multiples will reinforce a given period, but intermixed period differences will not be equal to a rational multiple of either number with probability one. The relative primeness of data generated by one period will backfill the missing elements for that period, whereas the data from two different periods will become Weyl flat (see Definition 4). Theoretically, if we allowed the process to go on indefinitely, it would become ergodic, with the sets of differences from different periods becoming a set of full measure, while the set of differences from the same period remaining as a set of measure zero (see Theorem 3). Using these periods, we then deinterleave the processes. We demonstrate the EQUIMEA on three different data sets, all with ≈90% missing observations—a sparse single period set (Figures 1 and 2), a sparse set with two periods analyzed then deinterleaved (Figures 3–5), and a sparse set with three periods analyzed then deinterleaved (Figures 6–8).

The paper is organized as follows. The MEA is developed in Section 2. This includes the mathematical justifications for the procedure, computer simulations, and a discussion of how to estimate other components of the data. Section 2.2 includes the proof of Theorem 1, the main justification of the MEA. Section 3 gives a description and analysis of the EQUIMEA, including the deinterleaving procedure. Weyl’s equidistribution theorem plays a key role in this work (Theorem 3), but we point out again that the “engine” for the EQUIMEA is the MEA. Three simulations of the EQUIMEA are given in Section 4, including the deinterleaving of data sets with multiple periods.

We finish with some observations, discussing how the structure of these algorithms naturally leads to questions about mathematical modeling, computational science, and data science.

2. Data Sets with a Single Period

We first assume that the data has a single underlying period. We model the event times as a finite set of positive real numbers.

$$S={\left\{s_j\right\}}_{j=1}^n,\mathrm{with}{s}_j=k_j\tau +\phi +{\eta }_j,$$

(1)

where the period $\tau$ is a fixed positive real number, the $k_j$’s are non-repeating positive integers, the phase $\phi$ is a real random variable uniformly distributed over the interval $[0,\tau )$ and the additive noise elements $\left\{{\eta }_j\right\}$ are zero-mean independent and identically distributed (iid) error terms. We refer to $\tau$ as the generator of the process. We also assume that the ${\eta }_j$’s have a symmetric probability density function (pdf) and that $|{\eta }_j|<{\displaystyle \frac{\tau }{2}}$ for all j. The assumption that $|{\eta }_j|<{\displaystyle \frac{\tau }{2}}$ for all j is important, for this does not allow noisy data elements to cross over each other. This crossover could, in effect, create false periods.

By assuming that the $k_j$’s are increasing, we can interpret the $s_j$’s as an increasing sequence of event times with time gaps determined by the $k_j$’s. Given S and a value of $\tau$ that is an underlying period of S, the numbers ${\displaystyle \frac{\tau }{2}},{\displaystyle \frac{\tau }{3}},\dots$ are also periods. We refer to these values as the harmonics of τ.

We can think of the event times as the set of zero-crossings of a sinusoid. If $f\left(t\right)=\sin ({\displaystyle \frac{\pi }{\tau }}t-\phi ),$ then S is a finite subset of the zero-crossings of f with missing observations and small perturbations in the zero-crossings (jitter noise). Given a set of event times with an underlying period, the signal processing community usually models event times with a set of Dirac deltas called a pulse train. Work on the analysis of event times includes other methods, including signal sampling, periodogram-based methods for spectral analysis of the pulse train, filtering, and binning and searching. The papers [2,3,4,15] contain numerous references to the signal processing literature.

The MEA extracts an estimate of $\tau$. The procedure is computationally efficient and requires little input data. Given reasonable data, it quickly converges with very high probability to either the exact value of $\tau$ (when the data is noise-free) or an estimate $\widehat{\tau }$ (when the data has additive noise). When there is no noise, the MEA produces $\tau$ with very high probability even if we are given very few (e.g., $n=10$) data elements, independent of the number of missing measurements. In the presence of noise (non-zero ${\eta }_j$’s in (1)) and false data (or outliers), there is a tradeoff between the number of data samples, the amount of noise and the percentage of outliers. The algorithm performs well given low noise for $n\ge 10$, but will degrade as noise is increased. There is always a trade-off between the amount of data and the amount of noise. Given more data, various statistical techniques ([2,3,4,15]) can be used to reduce noise effects and speed up convergence. The EQUIMEA, although more complicated, can also be used to extract $\tau$.

2.1. The Modified Euclidean Algorithm (MEA)

Assume that we have a set of event times S (as in (1)). The MEA is a procedure for finding $\tau$. The mathematical justifications for the procedure use number theory. We cite Hardy and Wright [16], Ireland and Rosen [17], Knuth [18,19,20], Leveque [21], Rosen [22], and Schroeder [23] as number theory references.

First, we recall the Euclidean algorithm. (Knuth calls the Euclidean algorithm “the grandfather of all algorithms” because it is the oldest algorithm still in use today [19], p. 318.) Given two positive integers a and b, $b<a$, we say that b divides a if there exits a positive integer k such that $a=k·b$. This is denoted by $b|a$. An integer $p>1$ that has no divisors other than 1 and itself is a prime. An integer $m>1$ that is not prime is called a composite. Prime numbers are the fundamental building blocks of the integers. The Fundamental Theorem of Arithmetic (see Rosen [22], p. 97) states that every positive integer $\ge 2$ is a product of powers of primes, unique up to the ordering of the primes.

The Euclidean algorithm is based on the property that $\mathbb{Z}$ is a Euclidean domain, i.e., given two positive integers a and b, $a>b$, there exist two positive integers q and r such that $a=q·b+r,0\le r<b.$ If $r=0$, then b divides a. This property can be used to develop the Euclidean algorithm, which can be represented in the following. Let “⟵” denote replacement, e.g., “$a⟵b$” means that the value of the variable a is to be replaced by the current value of the variable b. Given $a,b$, $a>b$, proceed as follows:

(1.) 

$a=b·q+r:0\le r<b$.

(2.) 

The algorithm terminates if $r=0$. Set $(a,b)=b$.

(3.) 

Else, set $a⟵b$ and $b⟵r$. Go to (1.).

The procedure yields the greatest common divisor of a and b. The greatest common divisor is the product of all the powers of prime factors p that divide both a and b. Note that $\gcd (k_1,\dots ,k_n)$, the greatest common divisor of the set $\left\{k_j\right\}$, is not the pairwise gcd of the set $\left\{k_j\right\}$. If $\gcd (k_1,\dots ,k_n)=1$, the set $\left\{k_j\right\}$ is called mutually relatively prime. If $\gcd (k_i,k_j)=1$ for all $i\ne j$, the set $\left\{k_j\right\}$ is called pairwise relatively prime. If a set is pairwise relatively prime, it is mutually relatively prime. However, the converse is not true. For example, consider the set $\{15,21,35\}$. Note that no pair in this set is relatively prime, but the entire set is mutually relatively prime.

To work with our data sets, we have to make modifications to the standard Euclidean algorithm. We have data sets with more than two elements. The gcd of a set of more than two integers can be computed using Proposition 1(i.). We also have data sets that possibly have non-integer periods. Proposition 1(ii.) extends the gcd to multiples of a fixed real number $\tau >0$.

Proposition 1. 

$$(i.)\gcd (k_1,\dots ,k_n)=\gcd (k_1,\dots ,k_{n-2},\left(\gcd (k_{n-1},k_n)\right)).$$

$$(ii.)\gcd (k_1\tau ,\dots ,k_n\tau )=\tau \gcd (k_1,\dots ,k_n).$$

Proof. 

See Leveque [21], p. 16.    □

Let $\gcd (k_1,\dots ,k_n)=\kappa$, and so $\gcd (k_1\tau ,\dots ,k_n\tau )=\kappa \tau$. Then, $\kappa \tau$ plays the role of the fundamental unit of the set $S=\{k_1\tau ,\dots ,k_n\tau \}$. We say that the elements in the set S are commensurate to $\kappa \tau$, i.e., every element in the set S can be expressed as an integer multiple of $\kappa \tau$. For example, the elements of the set $\{{\displaystyle \frac{3}{7}},1{\displaystyle \frac{4}{7}},4\}$ are commensurate to ${\displaystyle \frac{1}{7}}$. Note, $\gcd (3,11,28)=1$. If no such fundamental unit exists, we call the elements of the set incommensurate.

Remark 2. 

All finite sets of rational numbers are commensurate. Add a single irrational number to a finite set of rational numbers and this new set is incommensurate.

The first step of the Euclidean algorithm involves division. Noisy data makes this step unstable in the following sense. The additive noise components could be non-zero, but arbitrarily close to zero. Dividing by these numbers could result in arbitrarily large numbers. Our third modification of the Euclidean algorithm addresses this. We develop a form of the procedure based on sorting and subtraction rather than division. Although this requires additional iterations, it establishes the groundwork to modify the algorithm so that it is stable with respect to noise.

Remark 3. 

It is important to note that Proposition 2 plays a fundamental role in both the MEA and EQUIMEA.

Proposition 2. 

$$\gcd (k_1,\dots ,k_n)=\gcd ((k_1-k_2),(k_2-k_3),\dots ,(k_{n-1}-k_n),k_n).$$

Proof. 

First assume that $\gamma$ is a positive integer that divides each $k_j$, i.e., $\gamma |k_j$ for $j=1,\dots ,n$. Then, $\gamma \left|\right(k_j-k_{j+1})$ for $j=1,\dots ,n-1$, and $\gamma |k_n$. Therefore, $\gamma$ is a divisor of $((k_1-k_2),(k_2-k_3),\dots ,(k_{n-1}-k_n),k_n)$. Conversely, assume $\delta$ is a positive integer such that $\delta \left|\right(k_j-k_{j+1})$ for $j=1,\dots ,n-1$, and $\delta |k_n$. Therefore, there exist positive integers c and d such that $c\delta =k_n$ and $d\delta =(k_{n-1}-k_n)$. Thus, $d\delta +k_n=(d+c)\delta =k_{n-1}$, and so $\delta |k_{n-1}$. Continuing in this fashion, we get that $\delta |k_j$ for $j=1,\dots ,n$. Therefore, since the sets $\left\{k_j\right\}$ and $\left\{(k_j-k_{j+1})\right\}\cup \left\{k_n\right\}$ have the same divisors, $\gcd (k_1,\dots ,k_n)=\gcd ((k_1-k_2),(k_2-k_3),\dots ,(k_{n-1}-k_n),k_n).$    □

Remark 4. 

After the first sort and subtraction, the gcd of the integers is invariant throughout the remainder of the process.

In order to eliminate the phase information $\phi$, we subtract it. This is justified by the following.

Proposition 3. 

If

$$\beta =\gcd ((k_1-k_2),(k_2-k_3),\dots ,(k_{n-1}-k_n)),$$

then $\beta \left|\right(k_l-k_m)$ for all $1\le l<m\le n$.

Proof. 

Let $\beta =\gcd ((k_1-k_2),(k_2-k_3),\dots ,(k_{n-1}-k_n))$. Then, since $\beta \left|\right(k_1-k_2)$ and $\beta \left|\right(k_2-k_3)$ there exist positive integers c and d such that $c\beta =(k_1-k_2)$ and $d\beta =(k_2-k_3)$. Adding gives $(c+d)\beta =(k_1-k_3)$. Continuing in this fashion gives the result for $l=1$. Shifting to $l>1$ gives the general result.    □

We call the procedure the modified Euclidean algorithm (MEA). We first sort the $s_j$’s in descending order to allow for a more straightforward implementation of our algorithm, i.e., $s_1\ge s_2\ge \cdots \ge s_n$. (Thus $k_n$ is the minimum of $\left\{k_j\right\}$.) We form a new set by subtracting adjacent pairs of these numbers, given by $s_j-s_{j+1}$. After this first operation, the phase information has been subtracted out and the resulting set has the simpler form

$$S^{\prime }={\left\{s_j\right\}}_{j=1}^{n-1},\mathrm{with}{s}_j=K_j\tau +{\eta }_j^{\prime },$$

where $K_j=k_j-k_{j+1}$ and ${\eta }_j^{\prime }={\eta }_j-{\eta }_{j+1}$. In subsequent iterations of the algorithm, the data maintains this same general form. Eliminating the phase information simplifies the process. The main theorem of the MEA, Theorem 1, applies to $\left\{K_j\right\}$ after $\phi$ has been eliminated.

$$\underline{\mathbf{\text{The}} \mathbf{\text{Modified}} \mathbf{\text{Euclidean}} \mathbf{\text{Algorithm}} \mathbf{\text{(MEA)}}}$$

Initialize: Sort the elements of S in descending order. Set $\mathtt{iter}=\mathtt{0}$ and import ${\eta }_0$.

(1.) 

[Adjoin 0 after first iteration.] If $\mathtt{iter}>\mathtt{0}$, then $S⟵S\cup \left\{0\right\}$.

(2.) 

[Form the new set with elements $(s_j-s_{j+1})$.] Set $s_j⟵(s_j-s_{j+1})$.

(3.) 

[Sort.] Sort the elements in descending order.

(4.) 

[Eliminate noise.] If $0\le s_j\le {\eta }_0$, then $S⟵S\setminus \left\{s_j\right\}$.

(5.) 

[Terminate or loop.] The algorithm terminates if S has only one element $s_1$. Declare $\widehat{\tau }=s_1$. If not, then set $\mathtt{iter}⟵(\mathtt{iter}+\mathtt{1})$. Go to (1.).

We then sort, subtract adjacent pairs and, after the first iteration, eliminate noise and adjoin the previous non-zero minimum to the set. The algorithm is continued by iterating this process of sorting, subtracting and eliminating the elements in $[0,{\eta }_0]$, adjoining the previous non-zero minimum at each new iteration, and terminating when only a single element remains. Note that Proposition 2 guarantees that the $\gcd (K_1,\dots ,K_{(n-1)})$ remains unchanged. The lone remaining element is equal to $\gcd (K_1,\dots ,K_{(n-1)})·\tau +\mathrm{error}\mathrm{term},$ where the error term is the result of the noise terms ${\eta }_j$ after several iterations of the MEA and the noise floor ${\eta }_0$.

2.1.1. The Noise Floor ${\eta }_0$

The additive noise elements create a need for a noise floor ${\eta }_0$. The $\left\{{\eta }_j\right\}$ are zero-mean independent identically distributed (iid) error terms. We assume that the ${\eta }_j$’s have a symmetric probability density function (pdf) and that $|{\eta }_j|<{\displaystyle \frac{\tau }{2}}$ for all j. The assumption that $|{\eta }_j|<{\displaystyle \frac{\tau }{2}}$ for all j does not allow noisy data elements to cross over each other, and thus creating false periods.

To deal with the noise, we establish a noise threshold ${\eta }_0$. After the first sort and subtract (which eliminates the phase $\phi$), we “zero out” the noise by eliminating the elements in $[0,{\eta }_0]$. Setting this noise floor parameter ${\eta }_0$ is key. If we know an estimate of the noise in the data, we set ${\eta }_0$ as twice the maximum range of the noise.

In general, we do not have an estimate of ${\eta }_0$. For unknown noise models, this estimate can be tricky. First, after the first iteration, the differencing operation has removed the independence of the error terms. Second, the ordering operation makes the nature of the dependence in subsequent iterations difficult to determine. Analysis of order statistics very often rests on an iid assumption, e.g., see Sarhan and Greenberg [24] and Reiss [25]. Without the iid assumption, this analysis leads to many open questions (see Reiss [25]). In general, beyond the first iteration, the pdf of the subsequent error terms becomes asymmetric, even when starting with iid ${\eta }_j$’s with symmetric pdf $f_{\eta }\left(\eta \right)$. This occurs due to the reordering before differencing at each iteration, and because after the first iteration, the errors are no longer iid.

We developed the following method of estimating ${\eta }_0$ in [4]. Suppose the pdf of the ${\eta }_j$’s is given by $f_{\eta }\left(\eta \right)$, and consider the set of differences obtained in the first iteration, given by

$$y_j=s_j-s_{j+1}=(k_j-k_{j+1})\tau +({\eta }_j-{\eta }_{j+1}).$$

(2)

Invoking the zero-mean iid assumption on the ${\eta }_j$’s, the pdf of $({\eta }_j-{\eta }_{j+1})$ is given by the convolution $f_{\eta }\left(\eta \right)\ast f_{\eta }\left(\eta \right)$. So, for example, if $f_{\eta }\left(\eta \right)\sim \mathcal{U}[-{\displaystyle \frac{\Delta }{2}},{\displaystyle \frac{\Delta }{2}}]$ ($\eta$ is uniformly distributed with parameter $\Delta$) then $f_{y_j}\left(y\right)=\mathrm{tri}[y-(k_j-k_{j+1})\tau ]$, the triangle function centered at $(k_j-k_{j+1})\tau$. A straightforward method for clustering the data is to employ a gradient operator to determine when a step has occurred. After the first iteration, the gradient is estimated, with large gradient values indicating a step or “edge” in the data. We have employed a simple estimator by convolving with an impulse response given by $[-1,0,1]$. The gradient operator has the effect of binning the data. Each bin gives an estimate ${\eta }_0$, which is given by the largest data point $s_k$ in the bin minus the smallest $s_j$ in the bin. Let ${\eta }_0$ equal twice the maximum estimate over all of the bins.

This binning process also yielded a very quick method of estimating $\tau$. After subtracting out the phase $\phi$ and binning after applying the gradient operator, simply average across the elements in each bin. and then apply the MEA. This led to a multistep estimating process which achieved the Cramér–Rao bound (CRB). We then developed a multi-step procedure that

(i).

Estimates $\tau$,

(ii).

Estimates the $k_j$’s, and then

(iii).

Refines the estimate of $\tau$ using the estimated $k_j$’s in a least-squares solution.

In ([4], p. 2291), we refer to this procedure as the MEA-LS. An extensive comparative analysis was presented in [4], pp. 2296–2298), Figures 1–4. Each of these figures presents a comparative performance analysis of MEA, MEA-LS, the 1024 point periodogram, and the 4096 point periodogram against the Cramér–Rao bound as the signal-to-noise ratio (SNR) went from 0 to 50. Figure 5 from ([4], p. 2299) did a similar analysis for increasing jitter noise, as jitter increased from 5% to 35%.

2.1.2. Connection with Analytic Number Theory

Theorem 1 and its corollaries give us that $\gcd (K_1,\dots ,K_{n-1})⟶1$ with probability 1 as $n⟶\infty$. Convergence is exponentially quick. Therefore, the modified Euclidean algorithm yields either the exact value of $\tau$ (when the data is noise-free) or an estimate $\widehat{\tau }$ (when the data has additive noise). In the noise-free case, the theory tells us that the algorithm very likely yields $\tau$ given as few as 10 data samples.

This is a manifestation of the structure of randomness over $\mathbb{Z}$. The key result is Theorem 1, which is proven by showing that in an arbitrarily large lattice of positive integers in n-dimensional Euclidean space, if we choose an element of the lattice, the probability that the n-tuple is relatively prime quickly converges to certainty as n increases. We cannot “randomly chose” positive integers $\{k_1,\dots ,k_n\}$, but we can choose the n-tuple as an element of a lattice in ${\mathbb{R}}^n$.

(1). 

Given a randomly chosen n-tuple $(k_1,\dots ,k_n)$$(n\ge 2)$ of positive integers in a finite symmetric lattice in ${\mathbb{R}}^n$,

$$P\{\gcd (k_1,\dots ,k_n)=1\}⟶1^{-}\mathrm{quickly}\mathrm{as}n⟶\infty .$$

(2). 

Moreover, we can compute P—

$$P\{\gcd (k_1,\dots ,k_n)=1\}⟶{\left[\zeta \left(n\right)\right]}^{-1}\mathrm{as}n⟶\infty ,,$$

where $\zeta \left(z\right)$ is Riemann’s zeta function.

The connection with the zeta function may at first seem surprising. However, if one looks at Euler’s product Formula (4), it is easier to see how the zeta function can play a role in understanding relative primeness. We discuss the zeta function in the following section. (See [1,2] for additional discussion.)

2.2. Pi, the Primes, and Probability

Riemann’s zeta function is defined in the complex numbers $\mathbb{C}$. Given a complex number $z=x+iy$ ($i^2=-1$, $x,y\in \mathbb{R}$), we say that x is the real part of z (denoted by $x=\Re z$) and y is the imaginary part of z (denoted by $y=\Im z$). Riemann’s zeta function is defined on the complex half plane $\{z\in \mathbb{C}:\Re (z)>1\}$ by

$$\zeta \left(z\right)={\displaystyle \sum _{n=1}^{\infty }}{n}^{-z}.$$

(3)

Let $\mathbb{P}=\{p_1,p_2,p_3,\dots \}=\{2,3,5,\dots \}$ be the set of all prime numbers. Euler connected the zeta function to the primes in 1736 by proving that

$$\zeta \left(z\right)=\prod _{p_j\in \mathbb{P}}{\displaystyle \frac{1}{1-{\left(p_j\right)}^{-z}}}$$

(4)

(see Conway [26], pp. 187–194). We will show that given n$(n\ge 2)$ “randomly chosen positive integers” $\{k_1,\dots ,k_n\}$, the probability that this n-tuple is relatively prime is expressed in terms of $1/\zeta \left(n\right)$. This result is key to the MEA.

In the following $P\{·\}$ denotes probability.

Given a randomly chosen n-tuple $(k_1,\dots ,k_n)$$(n\ge 2)$ of positive integers in a finite symmetric lattice in ${\mathbb{R}}^n$,

$$P\{\gcd (k_1,\dots ,k_n)=1\}⟶{\left[\zeta \left(n\right)\right]}^{-1}$$

as the size of the lattice $⟶\infty$.

Heuristically, we could argue as follows. Given randomly distributed positive integers, by the law of large numbers, about $1/2$ of them are even, and $1/3$ of them are multiples of three and $1/p$ are multiples of some prime p. Thus, given n independently chosen positive integers,

$$\begin{array}{cc}\hfill P\left\{p\right|k_1,p|k_2,\dots ,\mathrm{and}p|k_n\}& =\\ \hfill \left(\mathrm{Independence}\right)& \\ \hfill P\left\{p\right|k_1\}·P\left\{p\right|k_2\}·\cdots ·P\left\{p\right|k_n\}& =\\ \hfill 1/\left(p\right)·1/\left(p\right)·\cdots ·1/\left(p\right)& =\\ \hfill 1/{\left(p\right)}^n.& \end{array}$$

Therefore,

$$P\left\{p\right|k_1,p|k_2,\dots ,\mathrm{and}p\left|k_n\right\}=1-1/{\left(p\right)}^n.$$

Calculating this for all of the primes gives us that

$$P\{\gcd (k_1,\dots ,k_n)=1\}={\displaystyle \prod _{j=1}^{\infty }}1-1/{\left(p_j\right)}^n.$$

where $p_j$ is the $j^{\mathrm{th}}$ prime. In this last equation, we have used the fact that by The Fundamental Theorem of Arithmetic, the prime factor decomposition of any integer $k_i>1$ appears among the prime numbers $p_j$ raised to some power. By Euler’s formula,

$$\zeta \left(z\right)={\displaystyle \prod _{j=1}^{\infty }}{\displaystyle \frac{1}{1-{\left(p_j\right)}^{-z}}},\Re \left(z\right)>1.$$

Thus,

$$P\{\gcd (k_1,\dots ,k_n)=1\}=1/\left(\zeta \left(n\right)\right).$$

Remark 5. 

This heuristic argument breaks down on the first line. Any uniform distribution on the positive integers would have to be identically zero. The merit in the argument lies in the fact that it gives an indication of how the zeta function plays a role in the problem.

The formal proof is developed as follows. Let $\mathrm{card}\{·\}$ denote cardinality of the set $\{·\}$, and for $n\ge 2$, let ${\{1,\dots ,\ell \}}^n$ denote the sublattice of positive integers in ${\mathbb{R}}^n$ with coordinates c such that $1\le c\le \ell$. Therefore,

$$N_n(\ell )=\mathrm{card}\{(k_1,\dots ,k_n)\in {\{1,\dots ,\ell \}}^n:\gcd (k_1,\dots ,k_n)=1\}$$

is the number of relatively prime elements in ${\{1,\dots ,\ell \}}^n$. Let $P_n(\ell )$ be the probability that n positive integers chosen at random from $\{1,\dots ,\ell \}$ are relatively prime. Thus

$$P_n(\ell )={\displaystyle \frac{N_n(\ell )}{{\ell }^n}}.$$

We have that ${lim}_{\ell \to \infty }{\displaystyle \frac{N_n(\ell )}{{\ell }^n}}$ gives the asymptotic meaning of “randomly chosen” positive integers.

Theorem 1. 

Let

$$N_n(\ell )=\mathrm{card}\{(k_1,\dots ,k_n)\in {\{1,\dots ,\ell \}}^n:\gcd (k_1,\dots ,k_n)=1\}.$$

(5)

For $n\ge 2$, we have that

$$\underset{\ell \to \infty }{lim}{\displaystyle \frac{N_n(\ell )}{{\ell }^n}}={\left[\zeta \left(n\right)\right]}^{-1}.$$

(6)

We begin with the following lemma, which gives us a counting formula for $N_n(\ell )$ expressed in terms of primes and products of primes. Let $⌊x⌋$ denote the floor function of x, namely

$$⌊x⌋=\underset{k\le x}{max}\{k:k\in \mathbb{Z}\}.$$

Lemma 1. 

Let $N_n(\ell )=\mathrm{card}\{(k_1,\dots ,k_n)\in {\{1,\dots ,\ell \}}^n:\gcd (k_1,\dots ,k_n)=1\}$ is the number of relatively prime elements in ${\{1,\dots ,\ell \}}^n$. Then

$$N_n(\ell )={\ell }^n-\sum _{p_i}{\left(⌊{\displaystyle \frac{\ell }{p_i}}⌋\right)}^n+\sum _{p_i<p_j}{\left(⌊{\displaystyle \frac{\ell }{p_i·p_j}}⌋\right)}^n-\sum _{p_i<p_j<p_k}{\left(⌊{\displaystyle \frac{\ell }{p_i·p_j·p_k}}⌋\right)}^n+\cdots .$$

(7)

Proof of Lemma 1. 

Choose a prime number $p_i$. The number of integers in $\{1,\dots ,\ell \}$ such that $p_i$ divides an element of that set is $⌊{\displaystyle \frac{\ell }{p_i}}⌋$. (Note that it is possible to have $p_i>\ell$, because $⌊{\displaystyle \frac{\ell }{p_i}}⌋=0$.) Therefore, the number of n-tuples $(k_1,\dots ,k_n)$ contained in the lattice ${\{1,\dots ,\ell \}}^n$ such that $p_i$ divides every integer in the n-tuple is

$${\left(⌊{\displaystyle \frac{\ell }{p_i}}⌋\right)}^n.$$

Next, if $p_i·p_j$ divides an integer k, then $p_i|k$ and $p_j|k$. Therefore, the number of n-tuples $(k_1,\dots ,k_n)$ contained in the lattice ${\{1,\dots ,\ell \}}^n$ such that $p_i$ or $p_j$ or their product divide every integer in the n-tuple is

$${\left(⌊{\displaystyle \frac{\ell }{p_i}}⌋\right)}^n+{\left(⌊{\displaystyle \frac{\ell }{p_j}}⌋\right)}^n-{\left(⌊{\displaystyle \frac{\ell }{p_i·p_j}}⌋\right)}^n,$$

where the last term is subtracted so that we do not count the same numbers twice (in a simple application of the inclusion–exclusion principle).

Continuing in this fashion, for three integers, say $p_i<p_j<p_k$, the number of n-tuples $(k_1,\dots ,k_n)$ contained in the lattice ${\{1,\dots ,\ell \}}^n$ such that $p_i$, $p_j$, or $p_k$ or any of their products divide every integer in the n-tuple is given by the inclusion–exclusion principle as

$$\begin{array}{cc} & \left[{\left(⌊{\displaystyle \frac{\ell }{p_i}}⌋\right)}^n+{\left(⌊{\displaystyle \frac{\ell }{p_j}}⌋\right)}^n+{\left(⌊{\displaystyle \frac{\ell }{p_k}}⌋\right)}^n\right]\hfill \\ -& \left[\left[{\left(⌊{\displaystyle \frac{\ell }{p_i·p_j}}⌋\right)}^n+{\left(⌊{\displaystyle \frac{\ell }{p_i·p_k}}⌋\right)}^n+{\left(⌊{\displaystyle \frac{\ell }{p_j·p_k}}⌋\right)}^n\right]-{\left(⌊{\displaystyle \frac{\ell }{p_i·p_j·p_k}}⌋\right)}^n\right].\hfill \end{array}$$

We can therefore see by induction that the number of n-tuples $(k_1,\dots ,k_n)$ contained in the lattice ${\{1,\dots ,\ell \}}^n$ such that $p_i,p_j,p_k,\dots$ or $p_l$ or any of their products divide every integer in the n-tuple is given by the inclusion–exclusion principle as

$$\sum _{p_i}{\left(⌊{\displaystyle \frac{\ell }{p_i}}⌋\right)}^n-\sum _{p_i<p_j}{\left(⌊{\displaystyle \frac{\ell }{p_i·p_j}}⌋\right)}^n+\sum _{p_i<p_j<p_k}{\left(⌊{\displaystyle \frac{\ell }{p_i·p_j·p_k}}⌋\right)}^n-\cdots .$$

But this counts the complement of $N_n(\ell )$ in the lattice ${\{1,\dots ,\ell \}}^n$. Therefore,

$$N_n(\ell )={\ell }^n-\sum _{p_i}{\left(⌊{\displaystyle \frac{\ell }{p_i}}⌋\right)}^n+\sum _{p_i<p_j}{\left(⌊{\displaystyle \frac{\ell }{p_i·p_j}}⌋\right)}^n-\sum _{p_i<p_j<p_k}{\left(⌊{\displaystyle \frac{\ell }{p_i·p_j·p_k}}⌋\right)}^n+\cdots .$$

This completes the proof of Lemma 1.    □

Proof of Theorem 1. 

Lemma 1 gives us that if $N_n(\ell )=\mathrm{card}\{(k_1,\dots ,k_n)\in {\{1,\dots ,\ell \}}^n:\gcd (k_1,\dots ,k_n)=1\}$ is the number of relatively prime elements in ${\{1,\dots ,\ell \}}^n$, then

$$N_n(\ell )={\ell }^n-\sum _{p_i}{\left(⌊{\displaystyle \frac{\ell }{p_i}}⌋\right)}^n+\sum _{p_i<p_j}{\left(⌊{\displaystyle \frac{\ell }{p_i·p_j}}⌋\right)}^n-\sum _{p_i<p_j<p_k}{\left(⌊{\displaystyle \frac{\ell }{p_i·p_j·p_k}}⌋\right)}^n+\cdots .$$

We now observe that

$${\displaystyle \frac{1}{{\ell }^n}}\sum _{p_i<p_j<\dots <p_k}{\left(⌊{\displaystyle \frac{\ell }{p_i·p_j·\cdots ·p_k}}⌋\right)}^n\le {\displaystyle \frac{1}{{\ell }^n}}\sum _{p_i<p_j<\dots <p_k\le \ell }{\left({\displaystyle \frac{\ell }{p_i·p_j·\cdots ·p_k}}\right)}^n=$$

$$\sum _{p_i<p_j<\dots <p_k\le \ell }{\left({\displaystyle \frac{1}{p_i·p_j·\cdots ·p_k}}\right)}^n={\left(\sum _{p\le \ell }{\displaystyle \frac{1}{p^n}}\right)}^k\le {\left(\sum _{p\mathrm{prime}}{\displaystyle \frac{1}{p^n}}\right)}^k\le {\left({\displaystyle \sum _{j=2}^{\infty }}{\displaystyle \frac{1}{j^n}}\right)}^k.$$

(8)

Since $n\ge 2$, this series is convergent. Thus, each term in the expansion of $N_n(\ell )/{\ell }^n$ is convergent. Now, let

$$M_k={\left({\displaystyle \sum _{j=2}^{\infty }}{\displaystyle \frac{1}{j^n}}\right)}^k,\mathrm{for}k=0,1,2,\dots .$$

(9)

By noting that since $n\ge 2$ and the sum is over $j\in \mathbb{N}\setminus \left\{1\right\}$, we get

$$0<\sum _j{\displaystyle \frac{1}{j^n}}\le \left({\displaystyle \frac{{\pi }^2}{6}}-1\right)<1.$$

(10)

Since the $k^{\mathrm{th}}$ term in the expansion of $N_n(\ell )/{\ell }^n$ is dominated by $M_k$ and since

$${\displaystyle \sum _{k=0}^{\infty }}{M}_k\le {\displaystyle \sum _{k=0}^{\infty }}{\left({\displaystyle \frac{{\pi }^2}{6}}-1\right)}^k={\displaystyle \frac{6}{(12-{\pi }^2)}}$$

(11)

is convergent, the series converges absolutely.

We now need the Möbius inversion function $\mu$, which is defined as follows. Let

$$\begin{array}{ccc}\mu \left(1\right)& =& 1,\hfill \\ \mu \left(m\right)& =& \left\{\begin{array}{cc}0\hfill & \mathrm{if}m\mathrm{is}\mathrm{divisible}\mathrm{by}\mathrm{the}\mathrm{square}\mathrm{of}\mathrm{a}\mathrm{prime},\hfill \\ {(-1)}^r\hfill & \mathrm{if}m=p_1·p_2·\cdots ·p_r,p_1,p_2,\dots ,p_r\mathrm{distinct}.\hfill \end{array}\right.\hfill \end{array}$$

(12)

The function $\mu$ is called an inversion function because if f is a function defined for all positive integers (an arithmetic function) and F is its sum over all divisors, i.e., $F\left(n\right)={\sum }_{d|n}f\left(d\right)$, then for all positive integers n, $\mu$ inverts f and F by

$$f\left(n\right)=\sum _{d|n}\mu \left(d\right)F(n/d)$$

(see Rosen [22], pp. 251–255). Euler showed that

$$\begin{array}{cc} & 1-\sum _{p_i}{\displaystyle \frac{1}{p_i^n}}+\sum _{p_i<p_j}{\displaystyle \frac{1}{{\left(p_i·p_j\right)}^n}}-\sum _{p_i<p_j<p_k}{\displaystyle \frac{1}{{\left(p_i·p_j·p_k\right)}^n}}+\cdots \hfill \\ =& \sum _m{\displaystyle \frac{\mu \left(m\right)}{m^n}}={\left[\zeta \left(n\right)\right]}^{-1}.\hfill \end{array}$$

(13)

where the last sum is over $m\in \mathbb{N}$. For $n\ge 2$, this series is absolutely convergent. This last equality follows because for $j,k,m,n\in \mathbb{N}$,

$$\sum _m{\displaystyle \frac{1}{m^n}}\sum _j{\displaystyle \frac{\mu \left(j\right)}{j^n}}=\sum _{m,j}{\displaystyle \frac{\mu \left(j\right)}{{\left(mj\right)}^n}}=\sum _k{\displaystyle \frac{1}{k^n}}\sum _{d|k}\mu \left(d\right)=1,$$

(14)

where we use the fact that both series in the first term converge absolutely and thus can be rearranged in any order (see Leveque [21], p. 120).

Now let $ϵ>0$ be given. We want to show that there exists $L>0$ such that for all $\ell >L$,

$$\left|{\displaystyle \frac{N_n(\ell )}{{\ell }^n}}-{\left[\zeta \left(n\right)\right]}^{-1}\right|<ϵ.$$

By (13), there exists $L_1>0$ such that for all $\ell >L_1$,

$$\left|\left[1-\sum _{p_i}{\displaystyle \frac{1}{p_i^n}}+\sum _{p_i<p_j}{\displaystyle \frac{1}{{\left(p_i·p_j\right)}^n}}-\sum _{p_i<p_j<p_k}{\displaystyle \frac{1}{{\left(p_i·p_j·p_k\right)}^n}}+\cdots \right]-{\left[\zeta \left(n\right)\right]}^{-1}\right|<{\displaystyle \frac{ϵ}{2}}.$$

(15)

We use the fact that for all $x\in \mathbb{R}$ and all $n\in \mathbb{Z}$, $⌊x⌋\le n$ if and only if $x<(n+1)$. Let

$$P_{\ell }=\sum _{\ell <p_i}{\displaystyle \frac{1}{p_i^n}}-\sum _{\ell <p_i<p_j}{\displaystyle \frac{1}{{\left(p_i·p_j\right)}^n}}+\sum _{\ell <p_i<p_j<p_k}{\displaystyle \frac{1}{{\left(p_i·p_j·p_k\right)}^n}}-\cdots .$$

Then

$$\left|{\displaystyle \frac{N_n(\ell )}{{\ell }^n}}-\left[1-\sum _{p_i}{\displaystyle \frac{1}{p_i^n}}+\sum _{p_i<p_j}{\displaystyle \frac{1}{{\left(p_i·p_j\right)}^n}}-\sum _{p_i<p_j<p_k}{\displaystyle \frac{1}{{\left(p_i·p_j·p_k\right)}^n}}+\cdots \right]\right|\le \left|P_{\ell }\right|.$$

(16)

But

$$\begin{array}{cc} & |P_{\ell }|\le {\displaystyle \sum _{m=(\ell +1)}^{\infty }}{\displaystyle \frac{1}{m^n}}\le {\displaystyle \sum _{m=(\ell +1)}^{\infty }}{\displaystyle \frac{1}{m^2}}\hfill \\ \le & {\displaystyle \sum _{m=(\ell +1)}^{\infty }}{\displaystyle \frac{1}{m(m-1)}}\le {\displaystyle \sum _{m=(\ell +1)}^{\infty }}\left[{\displaystyle \frac{1}{(m-1)}}-{\displaystyle \frac{1}{m}}\right]={\displaystyle \frac{1}{\ell }}.\hfill \end{array}$$

(17)

Choose $L_2$ such that for all $\ell >L_2$, ${\displaystyle \frac{1}{\ell }}<{\displaystyle \frac{ϵ}{2}}$. Let $L=max\{L_1,L_2\}$. Then, for all $\ell >L$

$$\begin{array}{cc} & \left|{\displaystyle \frac{N_n(\ell )}{{\ell }^n}}-{\left[\zeta \left(n\right)\right]}^{-1}\right|\hfill \\ =& \left|{\displaystyle \frac{N_n(\ell )}{{\ell }^n}}-\left[1-\sum _{p_i}{\displaystyle \frac{1}{p_i^n}}+\cdots \right]+\left[1-\sum _{p_i}{\displaystyle \frac{1}{p_i^n}}+\cdots \right]-{\left[\zeta \left(n\right)\right]}^{-1}\right|\hfill \\ \le & \left|{\displaystyle \frac{N_n(\ell )}{{\ell }^n}}-\left[1-\sum _{p_i}{\displaystyle \frac{1}{p_i^n}}+\cdots \right]\right|+\left|\left[1-\sum _{p_i}{\displaystyle \frac{1}{p_i^n}}+\cdots \right]-{\left[\zeta \left(n\right)\right]}^{-1}\right|\hfill \\ <& {\displaystyle \frac{ϵ}{2}}+{\displaystyle \frac{ϵ}{2}}=ϵ.\hfill \end{array}$$

(18)

This completes the proof of the theorem.    □

Corollary 1. 

Let $n\ge 2$. Given a randomly chosen n-tuple of positive integers $(k_1,\dots ,k_n)\in {\{1,\dots ,\ell \}}^n$, we have that

$$\underset{\ell \to \infty }{lim}{\displaystyle \frac{N_n(\ell )}{{\ell }^n}}={\left[\zeta \left(n\right)\right]}^{-1}.$$

Thus,

$$\gcd (k_1,\dots ,k_n)⟶1,$$

with probability ${\left[\zeta \left(n\right)\right]}^{-1}$ as $\ell ⟶\infty$.

Evaluating the zeta function, even at positive integer values, is challenging. Euler gave us a remarkable formula which evaluates $\zeta \left(z\right)$ at the even integers. Ireland and Rosen describe this as one of Euler’s “most remarkable computations” [17], p. 231. The exact evaluations of $\zeta$ at $\{3,5,7,\dots \}$ are still open (see, e.g., [27]). We list the values of $\zeta \left(z\right)$ for $z=2,4,6,8,\dots ,16$ in

Table 1. Some values of the Zeta Function $\zeta \left(n\right)$.

n	2	4	6	8	10	12	14	16
$\zeta \left(n\right)$	${\displaystyle \frac{{\pi }^2}{6}}$	${\displaystyle \frac{{\pi }^4}{90}}$	${\displaystyle \frac{{\pi }^6}{945}}$	${\displaystyle \frac{{\pi }^8}{9450}}$	${\displaystyle \frac{{\pi }^{10}}{93555}}$	${\displaystyle \frac{691{\pi }^{12}}{638512875}}$	${\displaystyle \frac{2{\pi }^{14}}{18243225}}$	${\displaystyle \frac{3617{\pi }^{16}}{325641566250}}$

.

We can, however, estimate $\zeta \left(n\right)$ at the odd integers. Moreover, the estimate shows that ${\left[\zeta \left(n\right)\right]}^{-1}⟶1$ quickly as n increases. In fact, the rate of convergence is exponential. This estimate, combined with Theorem 1, explains why as few as 10 data elements are needed to estimate $\tau$ as demonstrated in Table 2.

Proposition 4. 

Let $\omega \in (1,\infty )$. Then

$$\underset{\omega \to \infty }{lim}{\left[\zeta \left(\omega \right)\right]}^{-1}=1,$$

(19)

converging to 1 from below faster than $(1-2^{1-\omega })$.

Proof. 

Since

$$\zeta \left(\omega \right)={\displaystyle \sum _{n=1}^{\infty }}{n}^{-\omega }$$

and $\omega >1$,

$$\begin{array}{cc} & 1\le \zeta \left(\omega \right)=1+{\displaystyle \frac{1}{2^{\omega }}}+{\displaystyle \frac{1}{3^{\omega }}}+{\displaystyle \frac{1}{4^{\omega }}}+{\displaystyle \frac{1}{5^{\omega }}}+\cdots \hfill \\ \le & 1+{\displaystyle \frac{1}{2^{\omega }}}+{\displaystyle \frac{1}{2^{\omega }}}+\underset{4-\mathrm{times}}{\underbrace{{\displaystyle \frac{1}{4^{\omega }}}+\cdots +{\displaystyle \frac{1}{4^{\omega }}}}}+\underset{8-\mathrm{times}}{\underbrace{{\displaystyle \frac{1}{8^{\omega }}}+\cdots +{\displaystyle \frac{1}{8^{\omega }}}}}+\cdots \hfill \\ =& {\displaystyle \sum _{k=0}^{\infty }}{\left({\displaystyle \frac{2}{2^{\omega }}}\right)}^k={\displaystyle \frac{1}{1-\frac{2}{2^{\omega }}}}={\displaystyle \frac{1}{1-2^{1-\omega }}}.\hfill \end{array}$$

(20)

Thus,

$$(1-2^{1-\omega })\le {\left[\zeta \left(\omega \right)\right]}^{-1}<1,$$

(21)

and so

$${\left[\zeta \left(\omega \right)\right]}^{-1}⟶1^{-}\mathrm{as}\omega ⟶\infty ,$$

converging to 1 from below faster than $(1-2^{1-\omega })$.    □

The first step of the MEA eliminates the additive phase $\phi$. After this first step, the MEA is working with the differences of pairs of the initial data, having the form

$$K_1\tau ,\dots ,K_{(n-1)}\tau ,\mathrm{where}{K}_j=k_j-k_{(j+1)},j=1,\dots (n-1).$$

By Proposition 3, a prime $p_i$ divides each $k_j$ if and only if $p_i$ divides $k_j-k_n$, $j=1,\dots (n-1)$. This just shifts the point $(1,1,\dots ,1)$ to the point $((k_n+1),(k_n+1),\dots ,(k_n+1))$. Therefore, for $n\ge 3$, we have that

$$\underset{\ell \to \infty }{lim}{\displaystyle \frac{N_{(n-1)}(\ell ,k_n)}{{\ell }^{n-1}}}={\left[\zeta (n-1)\right]}^{-1}.$$

Combining Theorem 1 with Propositions 1 and 3 and inequality (21) shows that the algorithm generates the underlying period $\tau$ in the noise-free case as the number of data elements n goes to infinity. Moreover, (21) shows that the algorithm very likely produces this value in the noise-free case with as few as 10 data elements.

Corollary 2. 

Let $n\ge 3$. Given a randomly chosen n-tuple of positive integers $(k_1,\dots ,k_n)\in {\{1,\dots ,\ell \}}^n$ and a fixed positive real number τ,

$$\gcd (k_1\tau ,\dots ,k_{\left(n\right)}\tau )⟶\tau ,$$

with probability 1 as $n⟶\infty$.

2.3. Simulations of the MEA

We tested the MEA by designing computer simulations. We set up 100-loop Monte-Carlo runs, and then calculated statistics based on these computations. Let n denote the number of data elements in a given experiment, and without loss of generality, let $\tau =1$ in all experiments. This choice is arbitrary, but it does allow for a direct visual analysis of the results. Any other fixed real positive number would have yielded similar results.

Let $\widehat{\tau }$ denote the value the algorithm gives for $\tau$, and let $std\left(\widehat{\tau }\right)$ denote the experimental standard deviations. The initial phase $\phi$ was chosen randomly in $[0,\tau )$, and did not play a factor as it was eliminated after the first differencing. Noise values ${\eta }_j$ were modeled as uniformly distributed with the probability distribution function (pdf) $f_{\eta }\left(\eta \right)\sim \mathcal{U}[-{\displaystyle \frac{\Delta }{2}},{\displaystyle \frac{\Delta }{2}}]$, where $\mathcal{U}[\alpha ,\beta ]$ denotes the uniform distribution across the interval $[\alpha ,\beta ]$. Then, for example, $\Delta =2\times {10}^{-1}$ implies random phase jitter that is $\pm 10\%$ of the period $\tau =1$. In Table 3 various noise threshold values ${\eta }_0$ equaled $\Delta$. (We noted that an increase in the noise floor ${\eta }_0$ had the effect of speeding up the algorithm.)

The first set of simulations assumed that the data had no additive noise, i.e., for these simulations, ${\eta }_0=0$.

(1.) 

Estimation from data without additive noise.

The first simulation examined the effects that changes in n and in the percentage of missing observations have on the algorithm’s performance. The data points had no additive noise i.e., ${\eta }_j=0$ for all j. The algorithm converged in many cases to the exact value of $\tau =1$. When the number of events n was extremely low, the MEA also converged to multiples of $\tau$.

The missing data elements were modeled by creating jumps in the $k_j$’s as follows. We chose an integer l randomly from the interval $[1,M]$. Given $k_j$, $k_{j+1}=k_j+l$. Increasing M increased possible jumps, thus making the data increasingly sparse. Results are shown in Table 2. We let $\%miss$ denote the experimentally determined average percentage of missing observations and $iter$ denote the average number of iterations required to converge. To interpret these, again visualize the data as zero-crossings of $f\left(t\right)=\sin \left({\displaystyle \frac{\pi }{\tau }}t-\phi \right)$. A random process has removed $\%miss$ of the zero crossings of f, leaving only n observations.

The top half of Table 2 shows the effect of changing M and, therefore, changing the percentage of missing observations. Given insufficient data, the algorithm may converge to a multiple of $\tau$. Columns labeled $\tau$, $2\tau$, $3\tau$, and $\ge 4\tau$ indicate the percentage of runs that converged to these values. The algorithm is able to choose $\tau$ correctly based on $n=10$ data samples, even with $99.99988928\%$ of the possible observations missing. Convergence in the noise-free case depends on n but is independent of M, as shown by the analysis above. The bottom half of Table 2 illustrates the effect of changing n for M fixed. Reliable results are achieved for $n\ge 10$. Note, however, although it is very probable that as few as 10 data elements will produce $\tau$, it is still possible that one could get a multiple of $\tau$. We did get an outlier in one simulation, as one can see in the fourth line of the lower table.

Table 2 shows that given $n\ge 10$ event times in S, the MEA works very well (even with $99.99988928\%$ of the zero crossings removed). However, the algorithm breaks down as the number of elements in S is reduced below 10. This is consistent with the mathematical underpinnings of the MEA, as given by Theorem 1 and Proposition 4. We note that the result for four data elements is a bit low, and should be closer to $1/\zeta \left(4\right)=90/{\pi }^4$, whereas those for six and eight are closer to theoretical values of $1/\zeta \left(6\right)=945/{\pi }^6$ and $1/\zeta \left(8\right)=9450/{\pi }^8$, respectively.

Table 2. Modeling the MEA with noise-free data.

n	M	$\%\mathit{miss}$	iter	$\mathit{\tau }$	$2\mathit{\tau }$	$3\mathit{\tau }$	$4\mathit{\tau }$	≥$5\mathit{\tau }$
10	${10}^1$	$88.917$	$7.70$	$100\%$	0	0	0	0
10	${10}^2$	$98.878$	$47.58$	100	0	0	0	0
10	${10}^3$	$99.888$	$436.79$	100	0	0	0	0
10	${10}^4$	$99.989$	$2293.16$	100	0	0	0	0
10	${10}^5$	$99.999$	$\mathrm{30,167.68}$	100	0	0	0	0
4	${10}^2$	$94.754$	$11.38$	$88\%$	6	4	2	0
6	${10}^2$	$93.076$	$11.84$	97	3	0	0	0
8	${10}^2$	$90.853$	$9.25$	99	1	0	0	0
10	${10}^2$	$88.951$	$6.87$	99	1	0	0	0
12	${10}^2$	$87.100$	$6.74$	100	0	0	0	0
14	${10}^2$	$85.144$	$6.38$	100	0	0	0	0

Table 2. Modeling the MEA with noise-free data.

n	M	$\%\mathit{miss}$	iter	$\mathit{\tau }$	$2\mathit{\tau }$	$3\mathit{\tau }$	$4\mathit{\tau }$	≥$5\mathit{\tau }$
10	${10}^1$	$88.917$	$7.70$	$100\%$	0	0	0	0
10	${10}^2$	$98.878$	$47.58$	100	0	0	0	0
10	${10}^3$	$99.888$	$436.79$	100	0	0	0	0
10	${10}^4$	$99.989$	$2293.16$	100	0	0	0	0
10	${10}^5$	$99.999$	$\mathrm{30,167.68}$	100	0	0	0	0
4	${10}^2$	$94.754$	$11.38$	$88\%$	6	4	2	0
6	${10}^2$	$93.076$	$11.84$	97	3	0	0	0
8	${10}^2$	$90.853$	$9.25$	99	1	0	0	0
10	${10}^2$	$88.951$	$6.87$	99	1	0	0	0
12	${10}^2$	$87.100$	$6.74$	100	0	0	0	0
14	${10}^2$	$85.144$	$6.38$	100	0	0	0	0

(2.) 

Uniformly distributed noise.

We assume that the ${\eta }_j$’s have a uniform distribution, given by $f_{\eta }\left(\eta \right)\sim \mathcal{U}[-{\displaystyle \frac{\Delta }{2}},{\displaystyle \frac{\Delta }{2}}]$. The top half of Table 3 illustrates the effect of increasing M, resulting in more missing observations with a fixed noise parameter. Larger M generally requires more data to maintain the same accuracy in $\widehat{\tau }$ and results in larger $std\left(\widehat{\tau }\right)$. The bottom half shows the effect of increasing noise with n and M fixed. The noise floor was given by ${\eta }_0\ge \Delta$. This prevents noise from having noisy elements cross over each other. Again, if noisy elements cross over, this creates false periods.

Table 3. Modeling the MEA with noisy data.

n	M	Δ	% miss	iter	$\widehat{\mathit{\tau }}$	$\mathit{std}\left(\widehat{\mathit{\tau }}\right)$
10	${10}^1$	${10}^{-3}$	$89.123$	$7.31$	$0.9995$	$0.0002$
10	${10}^2$	${10}^{-3}$	$97.887$	$7.73$	$0.9980$	$0.0010$
50	${10}^3$	${10}^{-3}$	$99.803$	$11.24$	$0.9973$	$0.0035$
10	${10}^1$	${10}^{-3}$	$89.13$	$4.31$	$0.9995$	$0.0002$
10	${10}^1$	${10}^{-2}$	$87.94$	$4.45$	$0.9883$	$0.0051$
10	${10}^1$	${10}^{-1}$	$88.05$	$4.33$	$0.8857$	$0.0432$

Table 3. Modeling the MEA with noisy data.

n	M	Δ	% miss	iter	$\widehat{\mathit{\tau }}$	$\mathit{std}\left(\widehat{\mathit{\tau }}\right)$
10	${10}^1$	${10}^{-3}$	$89.123$	$7.31$	$0.9995$	$0.0002$
10	${10}^2$	${10}^{-3}$	$97.887$	$7.73$	$0.9980$	$0.0010$
50	${10}^3$	${10}^{-3}$	$99.803$	$11.24$	$0.9973$	$0.0035$
10	${10}^1$	${10}^{-3}$	$89.13$	$4.31$	$0.9995$	$0.0002$
10	${10}^1$	${10}^{-2}$	$87.94$	$4.45$	$0.9883$	$0.0051$
10	${10}^1$	${10}^{-1}$	$88.05$	$4.33$	$0.8857$	$0.0432$

Table 3 shows that the estimates of the period skew toward underestimating $\tau$. We again note that this leads to open questions involving order statistics. In the original data, the assumption is that the noise components ${\eta }_j$’s are independent identically distributed (iid). The first differencing removes iid, and the subsequent differencing and sorting then makes this noise negatively skewed. Statistical analysis of the noise after several steps of the MEA is an open question. Standard results in order statistics assume iid (e.g., see Sarhan and Greenberg [24] and Reiss [25]).

2.4. Comparison with Other Methods

Given noisy data from a periodic point process that has no missing observations, least squares procedures can be used to solve for maximum likelihood estimates of the period. Under more general conditions, Fourier analytic methods, e.g., Wiener’s periodogram, can be used to solve for estimates which are approximately maximum likelihood. However, these methods break down when the data has an increasing number of missing observations. Juxtaposed with these methods, the MEA provides parameter estimations that, while not being maximum likelihood, can be used as initialization in an algorithm that achieves the Cramér–Rao bound for moderate noise levels. We describe the conditions under which the least squares procedures and Fourier analytic methods do not produce estimates close to maximum likelihood, and show that the number theoretic methods provide a reliable estimate in these cases. We also discuss the type of data for which the number theoretic methods fail to produce good estimates.

2.4.1. Least Squares

In this section, we assume our data is in the form (1) with the additional assumption that ${\eta }_j$ is zero-mean additive white Gaussian noise. Initialize by sorting the elements of S in descending order and import ${\eta }_0$. We eliminate $\phi$ by forming the differences $y_j=t_{j+1}-t_j=(k_{j+1}-k_j)\tau +({\eta }_{j+1}-{\eta }_j)$, and clean out the elements in $[0,{\eta }_0]$. This yields

$$\left[\begin{array}{c}{y}_1\\ y_2\\ ⋮\\ y_{N-1}\end{array}\right]=\left[\begin{array}{c}{k}_1-k_2\\ k_2-k_3\\ ⋮\\ k_{N-1}-k_N\end{array}\right]\tau +\left[\begin{array}{c}{\delta }_1\\ {\delta }_2\\ ⋮\\ {\delta }_{N-1}\end{array}\right],$$

(22)

where ${\delta }_j={\eta }_{j+1}-{\eta }_j$. We may write (22) compactly in an obvious notation as

$$\mathbf{y}=X_d\tau +\mathit{\delta }.$$

(23)

Equation (23) is a linear regression problem whose least squares solutions yield the minimum-variance unbiased estimate when the noise is zero-mean Gaussian, e.g., see Kay [28]. Generally, use of (23) is preferred for estimating $\tau$, avoiding estimation of $\phi$ which has high variance, and so $\phi$ can have an arbitrary non-negative value. The solution to (23) corresponds to maximum likelihood estimation and takes the form of a least squares estimate

$$\widehat{\tau }={\left(X_d^T{R}_{\delta }^{-1}{X}_d\right)}^{-1}{X}_d^T{R}_{\delta }^{-1}\mathbf{y},$$

(24)

where $R_{\delta }=E\left[{\mathit{\delta }\mathit{\delta }}^T\right]$. We have assumed white noise so $R_{\delta }={\sigma }_v^2{\tilde{R}}_{\delta }$ where ${\tilde{R}}_{\delta }$ has 2’s on the main diagonal, $-1$’s on the first upper and lower diagonals, and zeros elsewhere. In general $R_{\delta }$ is full rank and its inverse can be expressed element-wise as ${\left[R_{\delta }^{-1}\right]}_{ij}=min(i,j)-ij/N$, and is therefore easily computed [28].

Although optimal, use of (24) requires knowledge of $X_d$. This is not a problem if there are no missing observations, for then $k_j=j$ for $j=1,2,\cdots N$. More generally, it also works with explicit knowledge of the $k_j$’s. However, when observations are arbitrarily missing, then the $k_j$’s are not known in general, and one is faced with more unknowns than equations in (23). We also need to note that if there were only a few missing observations, one could still use least squares by applying the procedure to data clusters in which the $k_j$’s are known, and then optimizing over these estimates.

2.4.2. Periodograms

For a general periodogram, let the data elements $t_j$ be thought of as recorded event times of a periodic procedure. This generates a zero-one time series or delta train with additive noise ${\eta }_j\left(t\right)$

$$s\left(t\right)={\displaystyle \sum _{j=1}^n}\delta (t-(k_j\tau +\phi ))+{\eta }_j\left(t\right).$$

An obvious approach to estimating the period $\tau$ is to circularly convolve s with $x(t,\tau )={\sum }_{k=1}^n\delta (t-\left(k\tau \right))$. This is equivalent to circularly applying a matched filter. This is not an optimal detector because noise manifests itself in the pulse, not the amplitude.

Another approach is to perform a spectral analysis of s using Wiener’s periodogram.

$${\mathbf{S}}_s\left(\omega \right)={\displaystyle \frac{1}{N}}{\left|{\displaystyle \sum _{j=1}^N}\exp \left(2\pi i{t}_j\omega \right)\right|}^2.$$

(25)

The highest peak of ${\mathbf{S}}_s\left(\omega \right)$ then gives an estimate of $\tau$, e.g., see Bartlett [29] and Brillinger [30]. This estimate has been shown to yield estimates of $\tau$ which are approximately maximum likelihood for small percentage of missing observations. However, this estimate degenerates as the percentage of missing observations exceeds 50–75%.

We now assume the noise ${\eta }_j$ has the pdf

$$f\left(\eta \right)={\displaystyle \frac{\exp (\alpha \cos (2\pi \eta /\tau )}{2\pi I_0\left(\alpha \right)}},\left|\eta \right|\le {\displaystyle \frac{\tau }{2}},$$

(26)

with parameter $\alpha ,$ where $I_0\left(\alpha \right)$ is the zero-th order Bessel function evaluated at $\alpha$. This pdf was used by Van Trees for analyzing phase-lock synchronization schemes. Note that $f\left(\eta \right)$ tends to a uniform pdf as $\alpha \to 0$, and tends to a Gaussian pdf as $\alpha \to \infty$. This is a fortuitous choice for the noise pdf, as it results in a maximum likelihood estimate of $\tau$ given by

$${\widehat{\tau }}_x={\mathrm{argmax}}_{\tau }{\left|{\displaystyle \sum _{j=1}^N}\exp \left(2\pi i{\displaystyle \frac{t_j}{\tau }}\right)\right|}^2.$$

(27)

(see Fogel and Gavish [31]).

This is seen as follows. First note that $f\left(\eta \right)=f(-\eta )$ for all $\alpha$ implies zero-mean. Assuming that $\beta ={[\phi ,\tau ]}^T$ is known, the pdf of $t_j$ given $\beta$ is

$$f\left(t_j\right|\mathit{\beta })=f(t_j-k_j\tau -\phi ),$$

(28)

and for iid noise,

$$f\left(\mathbf{t}\right|\beta )={\displaystyle \prod _{j=1}^N}f\left(t_j\right|\beta ).$$

(29)

The log-likelihood function is

$$L\left(\beta \right)=\ln f\left(\mathbf{t}\right|\beta )={\displaystyle \sum _{j=1}^N}[\alpha (\cos {\displaystyle \frac{2\pi }{\tau }}\left(t_j\right)-\phi )-\ln 2\pi I_0\left(\alpha \right)].$$

(30)

Therefore, the maximum likelihood estimate is found by maximizing $L\left(\beta \right)$ with respect to $\tau$, yielding

$${\widehat{\tau }}_{ML}={\mathrm{argmax}}_{\tau }{\displaystyle \sum _{j=1}^N}\cos \left(2\pi {\displaystyle \frac{(t_j-\phi )}{\tau }}\right).$$

(31)

Using the fact that

$$\cos {\displaystyle \frac{2\pi }{\tau }}(t_j-\phi )=\Re \left\{\exp \left(2\pi i{\displaystyle \frac{(t_j-\phi )}{\tau }}\right)\right\},$$

(32)

then, for $\varphi$ constant, maximizing $L\left(\beta \right)$ is equivalent to

$${\widehat{\tau }}_x={\mathrm{argmax}}_{\tau }{\left|{\displaystyle \sum _{j=1}^N}\exp \left(2\pi i{\displaystyle \frac{t_j}{\tau }}\right)\right|}^2,$$

(33)

which is in the form of spectrum analysis of a point process with occurrence times $t_j$.

These estimators depend directly on the exponential sum (as does the periodogram), and do not require knowledge of the pdf parameter $\alpha$. Use of (27) requires a fine search over $f=1/\tau$. This may result in a significant computational load because the FFT is not a good computational option, versus direct computation of the exponential sum, given a large number of zeros in $x\left(t\right)$. Moreover, estimates decay for >50–75% missing observations, and drop off for >90% missing observations.

2.4.3. The MEA

The MEA is designed to fill in the gap for >50–75% missing observations. It works when least squares and Weiner periodograms fail. The data can again be thought of as a set of event times of a periodic process, which generates a zero-one time series or delta train with additive noise $\eta \left(t\right)$

$$s\left(t\right)={\displaystyle \sum _{j=1}^n}\delta (t-(k_j\tau +\phi ))+{\eta }_j\left(t\right).$$

The $k_j$’s determine the best procedure for analyzing this data. Given a sequence of consecutive $k_j$’s or even an explicit knowledge of the $k_j$’s and assuming white Gaussian noise, we can get a maximum likelihood estimate using least squares. Fourier analytic methods work with some missing observations, but when the percentage of missing observations is too large (>50–75%), they break down. Number theoretic methods can work with very sparse data sets (>90% missing observations). However, here there is also a trade-off. Given 10 data elements with a high signal-to-noise ratio and no outliers, the number theoretic methods can produce reliable estimates even with $99\%$ missing observations. However, given an extremely noisy environment and a large number of outliers, the number theoretic methods require considerably larger data sets to get reliable estimates. If the period $\tau$ is much lower than the noise, this then creates false outliers, and the MEA will break down.

2.4.4. The MEA and the Periodogram: Missing Data

We finish this section with a comparison—an analysis of what happens to the MEA vs. the periodogram as the % of missing observations increases. The MEA is designed to work on all one-periodic data, but especially when the data becomes sparse. We create a data sets with no missing observations and no jitter noise and then relatively minimal (5% jitter) noise. We then, in sequence, we create data sets with 10%, 25%, 50%, 75% and finally 90% missing observations. We let $\tau =3.1415$ in all data sets. We set up 100-loop Monte-Carlo runs, and then calculated statistics based on these computations. These results are presented in

Table 4. Modeling the MEA with noise-free and 5% jitter data.

n	%   Missing	$\mathit{\tau }$ Noise-Free	$\mathit{\tau }$ 5% Jitter
200	0%	3.1415	3.1415
200	10%	3.1415	3.1413
200	25%	3.1415	3.1411
200	50%	3.1415	3.1408
200	75%	3.1415	3.1401
200	90%	3.1415	3.0103

. The values under ($\tau$ noise-free) and ($\tau$ 5% jitter) are averages.

Note the systematic drop in the data with 5% jitter. This is because of the increase in jitter noise as the data progressed through the algorithm (see Table 3 and the discussion following Table 3).

We then, using the same data, applied the periodogram. These results are presented in

Table 5. Modeling the MEA and the Periodogram with 5% jitter data.

n	%   Missing	MEA Within Error	Periodogram Within Error
200	0%	100%	100%
200	10%	100%	100%
200	25%	100%	100%
200	50%	100%	100%
200	75%	100%	64.3%
200	90%	95.63%	5.5%

. Here, we measured the range of accuracy by computing the % of trials within an error of 2% of $\tau =3.1415$.

Table 5 shows a relatively small systematic drop off for the MEA at 90% missing data, consistent with Table 3 and Table 4. In contrast, the periodogram performs very well up to ≈50–75% missing, and falls off for ≥90% missing. The periodogram produced peaks, but the height of the peaks varied with each simulation. It became impossible to determine the highest peak (See Section 2.4.2.)

2.5. Estimating Phase, Multiples, and Noise

Given an estimate $\widehat{\tau }$ of $\tau$, it is then possible to estimate the phase $\phi$, the multiples $k_j$, and the noise ${\eta }_j$ in the data. The phase estimation can be useful in the analysis of radar and sonar data, the multiples $k_j$ can be useful in the analysis of data with an associated shift register (such as hop times of frequency-hopping spread-spectrum (FHSS) radios), and information about the noise can be useful in algorithm analysis.

Phase estimation is important in radar and sonar analysis, and several researchers have several researchers have addressed this problem. Fogel and Gavish [31] produced an estimator $\widehat{\phi }$ for $\phi$, given by

$$\widehat{\phi }={\displaystyle \frac{\widehat{\tau }}{2\pi }}\arg \left\{{\displaystyle \sum _{j=1}^n}\exp \left(2\pi i{\displaystyle \frac{s_j}{\widehat{\tau }}}\right)\right\}.$$

Given a good estimate $\widehat{\tau }$ of $\tau$, we can derive $\widehat{\phi }$. We have that $\exp (2\pi i{\displaystyle \frac{k_j\tau }{\widehat{\tau }}})\approx 1$. For ${\eta }_j\ll \tau \approx \widehat{\tau }$, ${\eta }_j/\widehat{\tau }\ll 1$, and so $\exp (2\pi i{\eta }_j/\widehat{\tau })\approx \exp \left(0\right)=1$. Therefore,

$$\begin{array}{cc} & {\displaystyle \frac{\widehat{\tau }}{2\pi }}\arg \left\{{\displaystyle \sum _{j=1}^n}\exp (2\pi i{\displaystyle \frac{s_j}{\widehat{\tau }}})\right\}={\displaystyle \frac{\widehat{\tau }}{2\pi }}\arg \left\{{\displaystyle \sum _{j=1}^n}\exp (2\pi i{\displaystyle \frac{k_j\tau }{\widehat{\tau }}})\exp (2\pi i{\displaystyle \frac{{\eta }_j}{\widehat{\tau }}})\exp (2\pi i{\displaystyle \frac{\phi }{\widehat{\tau }}})\right\}\hfill \\ \approx & {\displaystyle \frac{\widehat{\tau }}{2\pi }}\arg \left\{{\displaystyle \sum _{j=1}^n}\exp (2\pi i{\displaystyle \frac{\phi }{\widehat{\tau }}})\right\}={\displaystyle \frac{\widehat{\tau }}{2\pi }}\arg \left\{n·\exp (2\pi i{\displaystyle \frac{\phi }{\widehat{\tau }}})\right\}\hfill \\ =& {\displaystyle \frac{\widehat{\tau }}{2\pi }}\left(\arg \left\{n\right\}+\arg \left\{\exp (2\pi i{\displaystyle \frac{\phi }{\widehat{\tau }}})\right\}\right)\hfill \\ =& {\displaystyle \frac{\widehat{\tau }}{2\pi }}\arg \left\{\exp (2\pi i{\displaystyle \frac{\phi }{\widehat{\tau }}})\right\}={\displaystyle \frac{\widehat{\tau }}{2\pi }}{\displaystyle \frac{2\pi \phi }{\widehat{\tau }}}=\phi .\hfill \end{array}$$

(34)

The MEA data after the first difference has the form

$${\sigma }^{\prime }={\{K_j\tau +{\eta }_j^{\prime }\}}_{j=1}^{n-1}\cup \{k_n\tau +\phi +{\eta }_n-\widehat{\phi }\},$$

(35)

where $K_j=k_j-k_{j+1}$ and ${\eta }_j^{\prime }={\eta }_j-{\eta }_{j+1}$. Let $⌊x+{\displaystyle \frac{1}{2}}⌋=\mathrm{round}\left(x\right)$ denotes rounding to the nearest integer. Given the estimate $\widehat{\tau }$, estimate $k_n$ by

$${\widehat{k}}_n=\mathrm{round}\left({\displaystyle \frac{k_n\tau +\phi +{\eta }_n-\widehat{\phi }}{\widehat{\tau }}}\right)$$

(36)

and $K_j$ by

$${\widehat{K}}_j=\mathrm{round}\left({\displaystyle \frac{K_j\tau +{\eta }_j^{\prime }}{\widehat{\tau }}}\right).$$

(37)

Then, ${\widehat{k}}_{n-1}={\widehat{K}}_{n-1}+{\widehat{k}}_n$, ${\widehat{k}}_{n-2}={\widehat{K}}_{n-2}+{\widehat{k}}_{n-1}$, and so on.

Estimating the noise ${\eta }_j$ can be approached as follows. If we have a sufficient amount of data elements, at each step of the MEA, data will be in separate clusters determined by the noise floor. We can bin the data by employing a gradient operator. In each bin, compute the average, and then subtract this value from to get the ${\eta }_j$ values. Then compute statistics on these values. This approach was used in [4], pp. 2293–2294, and had the effect of increasing the convergence rate of the algorithm and reducing the skewness in the estimate of $\tau$.

3. Data Sets with Multiple Periods

We now discuss the second algorithm, which gives the analysis and deinterleaving of multi-period pulse trains. We assume that the underlying periods are independent of each other, and therefore, with probability one, are not rational multiples of each other. However, even without this assumption, it is important to note that the algorithm will still produce results even with data that includes periods that are rational multiples.

The goal is to first identify the underlying periods (or “generators”) of data, and then to separate out the data that comes from a given generator. We model the data as the union of M copies of our previous data sets $\left\{S_i\right\}$, each with different underlying periods $\Gamma =\left\{{\tau }_i\right\}$, different $k_{ij}$’s, and different phases ${\phi }_i$. Our data is

$$S={\bigcup }_{i=1}^M{S}_i={\bigcup }_{i=1}^M{\{{\phi }_i+k_{ij}{\tau }_i+{\eta }_{ij}\}}_{j=1}^{n_i}.$$

(38)

In this formulation, $n_i$ is the number of elements from the $i^{th}$ generator, $\left\{k_{ij}\right\}$, for fixed i, is a linearly increasing sequence of natural numbers with missing observations, ${\phi }_i$ is a random variable uniformly distributed in $[0,{\tau }_i)$, and the ${\eta }_{ij}$’s are zero-mean iid Gaussian with standard deviation $3{\sigma }_{ij}<{\tau }_i/2$. The data models event times from M periodic processes. The data, after being sorted and then reindexed, has the form

$$S={\left\{s_l\right\}}_{l=1}^N=\{s_1,\dots ,s_N\},$$

(39)

where $N=\sum n_i$. The data will maintain this form throughout the algorithm.

We assume that the different periods or “generators” $\left\{{\tau }_i\right\}$ come from different processes. Therefore, we can assume that the difference of data elements

$$({\phi }_i+k_{ij}{\tau }_i+{\eta }_{ij})-({\phi }_{i^{\prime }}+k_{i^{\prime }{j}^{\prime }}{\tau }_{i^{\prime }}+{\eta }_{i^{\prime }{j}^{\prime }}),$$

for $i\ne i^{\prime }$, is incommensurate to ${\tau }_i$ and to ${\tau }_{i^{\prime }}$ with probability one. This follows from the assumption that the generators are independent from each other and that the rational numbers are a set of measure zero.

Remark 6. 

If two or more periods are commensurate, the algorithm will generate the fundamental period of the subset of commensurate periods. The phase information will then allow, almost surely, that these data sets can be deinterleaved.

We differ as in the MEA, but we compute and save all of the differences. We repeat this m times, saving the elements from each iteration. We form a union of all of these data elements. The relative primeness of data generated by one generator will backfill the missing elements for that generator (see Proposition 2), whereas the differenced data from two different generators will become Weyl flat.

We assume that we have knowledge of the range of $\left\{{\tau }_i\right\}$, namely lower and upper bounds $T_L$, $T_U$ such that $0<T_L\le {\tau }_i<T_U$. We then “phase wrap” the data as follows. Let $x\in \mathbb{R}$, let $\lfloor x\rfloor$ be the floor function, and let $〈x〉=x-\lfloor x\rfloor$, the fractional part of x. We define the mapping

$${\Phi }_{\rho }\left(s_l\right)=〈{\displaystyle \frac{s_l}{\rho }}〉={\displaystyle \frac{s_l}{\rho }}-⌊{\displaystyle \frac{s_l}{\rho }}⌋,$$

(40)

for $\rho \in [T_L,T_U)$. Therefore ${\Phi }_{\rho }\left(s_l\right)\in [0,1)$. Weyl’s Theorem applies asymptotically. We will show that for almost every choice of $\rho$ (in the sense of Lebesgue measure) ${\Phi }_{\rho }\left(s_l\right)$ is essentially uniformly distributed in the sense of Weyl.

This is a manifestation of the structure of randomness over a continuous interval $[T_L,T_U)$.

(1.) 

For almost every choice of $\rho$ (in the sense of Lebesgue measure), ${\Phi }_{\rho }\left(s_l\right)$ is essentially uniformly distributed in the sense of Weyl.

(2.) 

The set of $\rho$’s for which this is not true is rational multiples of $\left\{{\tau }_i\right\}$. Since the rational is countable and the finite union of countable sets is countable, these are a set of measure zero. Therefore, except for rational multiples of $\left\{{\tau }_i\right\}$, ${\Phi }_{\rho }\left(s_l\right)$ is essentially uniformly distributed in $[T_L,T_U)$.

Given $S_{iter}=\{s_1,\dots ,s_N\}$, we phase wrap the data by computing modulus of the spectrum, i.e., compute

$$|{\mathrm{Spec}}_{iter}\left(\tau \right)|=\left|{\displaystyle \sum _{l=1}^N}{e}^{(2\pi i{s}_l/\tau )}\right|.$$

(41)

The values of $|{\mathrm{Spec}}_{iter}\left(\tau \right)|$ will have peaks at the periods ${\tau }_j$ and, for $k>1$, the harmonics $\left({\tau }_j\right)/k$.

3.1. Weyl’s Equidistribution Theorem

Weyl’s equidistribution Theorem 2 and its extension Theorem 3 are the theoretical foundations of the EQUIMEA. They play key roles for the EQUIMEA, analogous to the role Theorem 1 played for the MEA.

The idea of Weyl’s Theorem is that given a fixed irrational number $\gamma$, the sequence of numbers

$$〈\gamma 〉,〈2\gamma 〉,\dots ,〈k\gamma 〉,\dots$$

is essentially uniformly scattered over $[0,1)$. This can be easily visualized using some tools from topological dynamics. Create a lattice $\left\{\right(m,n)$, $m,n\in \mathbb{N}\cup \left\{0\right\}\}$ in the upper right hand quadrant in ${\mathbb{R}}^2$. Now, draw a ray starting at $(0,0)$ making an angle $\theta$ with the positive x-axis. If $\theta$ is a rational number, then the ray intersects a lattice point $(m,n)$ (and infinitely many other lattice points $(km,kn),k\in \mathbb{N}$.) However, if $\theta$ is irrational, it never intersects with any lattice points, but does come arbitrarily close to infinitely many lattice points. (For example, if $\theta$ is the Golden Mean, given any $ϵ>0$, there are infinitely many pairs of adjacent Fibonacci numbers within $ϵ$ of the ray.) Now, consider the unit square with vertices $\left\{\right(0,0),(1,0),(1,1),(0,1\left)\right\}$. Glue the opposite sides together, resulting in a torus $\mathcal{T}$. If $\theta$ is rational, the ray on the torus will be a closed loop. If $\theta$ is irrational, the ray never closes. However, given any $ϵ>0$ and any point $t\in \mathcal{T}$, the ray will appear in the $ϵ$ disk around t infinitely often. In fact, the rate at which the ray “fills in” the disk is related to the Diophantine approximation of $\theta$. Very deep work of Katok, Stepin, Margulis et al. [32,33] addresses this.

Fourier series play a key role in Weyl’s Theorem. The definition, from Dym and McKean [34], follows. Let $\exp (·)=e^{(·)}$.

Definition 1 (Fourier Series). 

Let f be a periodic, absolutely and square integrable function on $\mathbb{R}$, with period $2\Psi$, i.e., $f\in L^1\cap L^2\left({\mathbb{T}}_{2\Psi }\right)$. The Fourier coefficients of f, $\widehat{f}\left[n\right]$, are defined by

$$\widehat{f}\left[n\right]={\displaystyle \frac{1}{2\Psi }}{\int }_{-\Psi }^{\Psi }f\left(t\right)\exp (-i\pi nt/\Psi )dt.$$

(42)

If $\{\widehat{f}\left[n\right]\}$ is absolutely and square summable ($\{\widehat{f}\left[n\right]\}\in l^1\cap l^2$), then the Fourier series of f is

$$f\left(t\right)=\sum _{n\in \mathbb{Z}}\widehat{f}\left[n\right]\exp (i\pi nt/\Psi ).$$

(43)

The space $L^2\left({\mathbb{T}}_{2\Psi }\right)$ is the canonical example of a separable Hilbert Space, and the set $\left\{\exp \right(-i\pi nt/\Psi ):n\in \mathbb{Z}\}$ is the canonical example of an orthonormal basis for $L^2\left({\mathbb{T}}_{2\Psi }\right)$.

Once again, let $\mathrm{card}\{·\}$ denote the cardinality of $\{·\}$.

Theorem 2 (Weyl). 

Given a fixed irrational number γ, then for every $a,b$ such that $0\le a<b<1$,

$$\underset{n\to \infty }{lim}{\displaystyle \frac{1}{n}}\mathrm{card}\{1\le k\le n:a\le 〈k\gamma 〉\le b\}=(b-a).$$

(44)

The theorem is strikingly intuitive. Weyl’s original proof used techniques and theory from the Fourier series. Our proof follows the development in Körner [35], which follows Weyl’s original development.

We first prove the following lemma.

Lemma 2. 

Let γ be a fixed irrational number, and let $a,b$ be real numbers such that $0\le a<b<1$. Let $f:\mathbb{T}⟶\mathbb{C}$ be a continuous function. Then

$$\underset{n\to \infty }{lim}{\displaystyle \frac{1}{n}}{\displaystyle \sum _{k=1}^n}f\left(2\pi k\gamma \right)={\displaystyle \frac{1}{2\pi }}{\int }_{-\pi }^{\pi }f\left(t\right)dt.$$

(45)

$$\underset{n\to \infty }{lim}{\displaystyle \frac{1}{n}}\mathrm{card}\{1\le k\le n:\left(2\pi k\gamma \right)\in [2\pi a,2\pi b]\}=(b-a).$$

(46)

Proof of Lemma 2. 

Let

$$G_n\left(f\right)={\displaystyle \frac{1}{n}}{\displaystyle \sum _{k=1}^n}f\left(2\pi k\gamma \right)-{\displaystyle \frac{1}{2\pi }}{\int }_{-\pi }^{\pi }f\left(t\right)dt.$$

(47)

We want to show that $G_n\left(f\right)⟶0$ as $n⟶\infty$. Note that

$$G_1\left(f\right)={\displaystyle \frac{1}{n}}{\displaystyle \sum _{k=1}^n}1-{\displaystyle \frac{1}{2\pi }}{\int }_{-\pi }^{\pi }1dt=0.$$

If $t\in [-\pi ,\pi ]$ and $m\in \mathbb{Z}$,

$$\begin{array}{ccc}\hfill |G_n\left(\exp \left(imt\right)\right)|& =& \left|{\displaystyle \frac{1}{n}}{\displaystyle \sum _{k=1}^n}\exp \left(2\pi ikm\gamma \right)-{\displaystyle \frac{1}{2\pi }}{\int }_{-\pi }^{\pi }\exp \left(imt\right)dt\right|\hfill \\ & =& \left|{\displaystyle \frac{1}{n}}\exp \left(2\pi km\gamma \right){\displaystyle \sum _{k=1}^{n-1}}\exp \left(2\pi ikm\gamma \right)-0\right|\hfill \\ & =& \left|{\displaystyle \frac{1}{n}}\exp \left(2\pi km\gamma \right)\left[{\displaystyle \frac{1-\exp \left(2\pi inm\gamma \right)}{1-\exp \left(2\pi im\gamma \right)}}\right]\right|\hfill \\ & =& {\displaystyle \frac{1}{n}}\left|\left[{\displaystyle \frac{1-\exp \left(2\pi inm\gamma \right)}{1-\exp \left(2\pi im\gamma \right)}}\right]\right|\hfill \\ & \le & {\displaystyle \frac{2}{n}}\left|\left[{\displaystyle \frac{1}{1-\exp \left(2\pi im\gamma \right)}}\right]\right|.\hfill \end{array}$$

(48)

Since $\gamma$ is irrational, $1-\exp \left(2\pi im\gamma \right)$ is never zero, and so

$${\displaystyle \frac{2}{n}}\left|\left[{\displaystyle \frac{1}{1-\exp \left(2\pi im\gamma \right)}}\right]\right|⟶0\mathrm{as}n⟶\infty .$$

(49)

Now, let $P\left(t\right)={\sum }_{m=-l}^l{a}_m\exp \left(imt\right)$ be any trigonometric polynomial. Then by linearity, and (48) and (49),

$$G_n\left(P\right)={\displaystyle \sum _{m=-l}^l}{a}_m{G}_n\left(\exp \left(imt\right)\right)⟶0\mathrm{as}n⟶\infty .$$

(50)

Now, let $g,h:\mathbb{T}⟶\mathbb{C}$ be two continuous functions. Let $ϵ>0$ be given. Assume that

$$\left|g\right(t)-h(t\left)\right|<ϵ$$

for all $t\in \mathbb{T}$. Then

$$\begin{array}{cc}|G_n\left(g\right)-G_n\left(h\right)|& \le {\displaystyle \frac{1}{n}}{\displaystyle \sum _{k=1}^n}|g\left(2\pi ikm\gamma \right)-h\left(2\pi ikm\gamma \right)|+{\displaystyle \frac{1}{2\pi }}{\int }_{-\pi }^{\pi }|g\left(t\right)-h\left(t\right)|dt\\ & <{\displaystyle \frac{1}{n}}nϵ+{\displaystyle \frac{1}{2\pi }}2\pi ϵ=ϵ+ϵ=2ϵ.\end{array}$$

(51)

We now use Weierstrass approximation (see [34,35]). By Weierstrass, there exists a trigonometric polynomial $P\left(t\right)$ such that, given any continuous function $f:\mathbb{T}⟶\mathbb{C}$, we have

$$\underset{t\in \mathbb{T}}{sup}|P\left(t\right)-f\left(t\right)|\le ϵ/3.$$

By (50) we can find N such that for all $n>N$,

$$|G_n\left(P\right)|<ϵ/3.$$

(52)

Thus, by (51),

$$|G_n\left(f\right)-G_n\left(P\right)|<2ϵ/3.$$

(53)

Therefore, by the triangle inequality, for all $n>N$,

$$|G_n\left(f\right)|\le |G_n\left(P\right)|+|G_n\left(f\right)-G_n\left(P\right)|<ϵ/3+2ϵ/3=ϵ.$$

(54)

To finish, we need two continuous partitions of unity $f_u$ and $f_l$ such that $f_u$ is one on $[2\pi a,2\pi b]$, zero outside of $[2\pi a-ϵ,2\pi b+ϵ]$, and $f_l$ is one on $[2\pi a+ϵ,2\pi b-ϵ]$, zero outside of $[2\pi a,2\pi b]$. By our computations above, there exists N such that for all $n>N$,

$$|G_n\left(f_u\right)|<ϵ,|G_n\left(f_l\right)|<ϵ.$$

Thus,

$$\begin{array}{cc} & {\displaystyle \frac{1}{2\pi }}{\int }_{-\pi }^{\pi }{f}_u\left(t\right)dt+ϵ\hfill \\ \ge & {\displaystyle \frac{1}{n}}\mathrm{card}\{1\le k\le n:\left(2\pi k\gamma \right)\in [2\pi a,2\pi b]\}\hfill \\ \ge & {\displaystyle \frac{1}{2\pi }}{\int }_{-\pi }^{\pi }{f}_l\left(t\right)dt-ϵ.\hfill \end{array}$$

(55)

Since $ϵ$ is arbitrary,

$${\displaystyle \frac{1}{n}}\mathrm{card}\{1\le k\le n:\left(2\pi k\gamma \right)\in [2\pi a,2\pi b]\}⟶(b-a)\mathrm{as}n⟶\infty .$$

(56)

This completes the proof of Lemma 2.    □

To finish the proof of Theorem 2, normalize the interval to $[0,1)$.

Remark 7. 

If γ is rational, i.e., there exist $k\in \mathbb{Z}$ and $n\in \mathbb{N}$ such that $\gamma =k/n$, then the Weyl equidistribution theorem is clearly false. For $m\in \mathbb{N}$, the sequence of values $〈m\gamma 〉$ is a finite set.

We finish this subsection by recalling some basic measure theory. A set X is a set of measure zero if, given any $ϵ>0$, we can cover the set with open sets of total length less than $ϵ$. Thus, any finite set $\{x_1,\dots ,x_n\}$ is of measure zero, for we can cover it with $\{(x_1-ϵ/4n,x_1+ϵ/4n),\dots ,(x_n-ϵ/4n,x_n+ϵ/4n)\}$. Similarly, any countable set $\{x_1,\dots ,x_n,\dots \}$ is of measure zero, for we can cover it with $\{(x_1-ϵ/4,x_1+ϵ/4),\dots ,(x_n-ϵ/4^n,x_n+ϵ/4^n),\dots \}$. The set of rationals $\mathbb{Q}$ is countable, and therefore of measure zero. Given an interval $[a,b]$ in $\mathbb{R}$, and two measurable functions $f,g$ defined on $[a,b]$, we say that $f=ga.e.$ ($a.e.$—almost everywhere) if they differ only on a set of measure zero.

The space of functions that are absolutely and square Lebesgue integrable functions on the real line are denoted by $L^1\cap L^2\left(\mathbb{R}\right)$. If f be a periodic, absolutely and square integrable function on $\mathbb{R}$, with period $2\Psi$, i.e., $f\in L^1\cap L^2\left({\mathbb{T}}_{2\Psi }\right)$, then f may be expanded in a Fourier series.

3.2. Extending Weyl to Measures

A given element for our data set has the form ${\phi }_i+k_{ij}{\tau }_i+{\eta }_{ij}$. The data elements are then mixed by an iterative process of sorting and subtraction. We again assume that the different periods or “generators” $\left\{{\tau }_i\right\}$ come from different processes. Therefore, we can assume that the difference of data elements

$$({\phi }_i+k_{ij}{\tau }_i+{\eta }_{ij})-({\phi }_{i^{\prime }}+k_{i^{\prime }{j}^{\prime }}{\tau }_{i^{\prime }}+{\eta }_{i^{\prime }{j}^{\prime }}),$$

for $i\ne i^{\prime }$, is incommensurate to both ${\tau }_i$ and ${\tau }_{i^{\prime }}$, i.e., is not equal to a rational multiple of either number. The relative primeness of data generated by one generator will backfill the missing elements for that generator, whereas the data from two different generators will become Weyl flat. Theoretically, if we allowed the process to go on indefinitely, it would become ergodic, with the sets of differences from different generators becoming a set of full measure, while the set of differences from the same generator remaining as a set of measure zero. Background for this section includes Blum and Mizel [36], Breiman [37], and Walters [38]. In particular, we base our discussion on Chapter 6 of Breiman [37] and Chapters 1–4 of Walters [38].

Recall that a probability space consists of a set $\mathbf{X}$, a collection $\mathcal{B}$ of Borel subsets of $\mathbf{X}$, and P, an additive measure normalized so that $P\left(\mathbf{X}\right)=\mathbf{1}$.

Definition 2. 

Suppose that $({\mathbf{X}}_1,{\mathcal{B}}_1,P_1)$ and $({\mathbf{X}}_2,{\mathcal{B}}_2,P_2)$ are probability spaces.

(a.)

A transformation $T:{\mathbf{X}}_1⟶{\mathbf{X}}_2$ is measurable if

$$B_2\in {\mathcal{B}}_2⟹T^{-1}\left(B_2\right)\in {\mathcal{B}}_1.$$

(b.)

A transformation $T:{\mathbf{X}}_1⟶{\mathbf{X}}_2$ is measure-preserving if T is measurable and

$$P_1\left(T^{-1}\left(B_2\right)\right)=P_2\left(B_2\right)for all B_2\in {\mathcal{B}}_2.$$

The term ergodic is an amalgamation of Greek words ergon (work) and odos (path). It was first used by Boltzmann to describe the action $T_{\theta }\left(t\right):t\in \mathbb{R}$ on an energy surface. Our discussion of the path of the ray on the torus $\mathcal{T}$ is relevant. The intuition is that we have an ergodic process if $\theta$ is irrational and do not have one if $\theta$ is rational.

Definition 3. 

Let $(\mathbf{X},\mathcal{B},P)$ be a probability space. A measure-preserving transformation T of $(\mathbf{X},\mathcal{B},P)$ is called ergodic if the only members B of $\mathcal{B}$ with the property that $T^{-1}\left(B\right)=B$ satisfy $P\left(B\right)=0$ or $P\left(B\right)=1$.

For a random variable, we say that a property holds almost surely if it holds except on a set of measure zero.

Definition 4. 

A sequence of real random variables $\left\{x_j\right\}\subset [0,1)$ is essentially uniformly distributed in the sense of Weyl if given $a,b$, $0\le a<b<1$, ${\displaystyle \frac{1}{n}}\mathrm{card}\{1\le j\le n:x_j\in [a,b]\}⟶(b-a)$ as $n⟶\infty$ almost surely.

Remark 8. 

We refer to data elements that become essentially uniformly distributed in the sense of Weyl as Weyl flat.

To apply this measure-theoretic variation of Weyl’s Theorem to our data sets, we have to assume that the data sets $\left\{S_i\right\}$ are not finite. The way to do this is to assume for each i, $\left\{k_{ij}\right\}$ is a linearly increasing infinite sequence of natural numbers with missing observations such that $k_{ij}⟶\infty$ as $j⟶\infty$. We must make this assumption because the result is only approximately true for a finite-length sequence.

Theorem 3. 

For almost every choice of ρ (in the sense of Lebesgue measure) ${\Phi }_{\rho }\left(s_l\right)$ is essentially uniformly distributed in the sense of Weyl.

Proof. 

Let $\rho \in [T_L,T_U]$, $\lfloor ·\rfloor$ be the floor function, and $〈·〉$ be the fractional part. The mapping

$${\Phi }_{\rho }\left(s_l\right)=〈{\displaystyle \frac{s_l}{\rho }}〉={\displaystyle \frac{s_l}{\rho }}-⌊{\displaystyle \frac{s_l}{\rho }}⌋,$$

(57)

is a measurable and measure-preserving mapping into $[0,1)$.

Claim. For a.e. choice of $\rho$, ${\Phi }_{\rho }$ is ergodic.    □

Proof of Claim. 

Normalize $[T_L,T_U)$ to $[0,1)$. Let X be a absolutely and square Lebesgue integrable random variable, i.e., $X\in L^1\cap L^2$. Therefore, we can expand X in a Fourier series

$$X\left(t\right)=\sum _{n\in \mathbb{Z}}\widehat{X}\left[n\right]\exp \left(i\pi nt\right),$$

(58)

with the Fourier coefficients $\widehat{X}\left[n\right]$ given by

$$\widehat{X}\left[n\right]={\int }_0^{1}X\left(t\right)\exp (-i\pi nt)dt.$$

(59)

Then

$$X\left({\Phi }_{\rho }\left(t\right)\right)=\sum _{n\in \mathbb{Z}}\widehat{X}\left[n\right]\exp (i\pi nt/\rho ).$$

(60)

For X to be invariant, for $a.e.t$,

$$\widehat{X}\left[n\right](1-\exp (i\pi nt/\rho ))=0\mathrm{for}\mathrm{all}n.$$

(61)

This implies either

$$\widehat{X}\left[n\right]=0\mathrm{or}\exp (i\pi nt/\rho )=1.$$

(62)

But since $\rho$ is irrational a.e., by Weyl’s equidistribution theorem,

$$\exp (i\pi nt/\rho )\ne 1$$

(63)

on a set of full measure.    □

Moreover, the set of $\rho$’s for which this is not true is rational multiples of $\left\{{\tau }_i\right\}$. Since $\mathbb{Q}$ is a countable set, and a finite union of countable sets is countable, the set of rational multiples of $\left\{{\tau }_i\right\}$ is countable, and therefore a set of measure zero. Thus, except for those values, ${\Phi }_{\rho }\left(s_l\right)$ is essentially uniformly distributed in $[0,1)$. The values at which ${\Phi }_{\rho }\left(s_l\right)=0$ almost surely are $\rho \in \{{\tau }_i/n:n\in \mathbb{N}\}$, which is a set of measure zero.

3.3. The EQUIMEA

The EQUIMEA algorithm is built upon the MEA, relying on the MEA procedure to backfill data elements from a given fixed generator. It has numerous applications [6]. The EQUIMEA uses Theorem 3 to separate out differenced elements from different generators. We differ as in the MEA (see Steps (1.)–(4.) of the EQUIMEA), but we compute and save all of the differences. We repeat this iter times, saving the elements from each iteration. We form a union of all of these data elements (see Step (5.)).

The set S is the union of the data generated by the M periodic processes, i.e., $S={\bigcup }_{i=1}^M{S}_i={\bigcup }_{i=1}^M{\{{\phi }_i+k_{ij}{\tau }_i+{\eta }_{ij}\}}_{j=1}^{n_i}$. Sort S in descending order and reindex. Let $\mathrm{card}\left\{S\right\}=N$, and let $\mathrm{Range}\left\{S\right\}=s_{\max }-s_{\min }$.

The relative primeness of data generated by one generator will “fill in” the missing elements for that generator (see Proposition 2). In contrast, the differenced data from two or more different generators will become Weyl flat.

We rely on a Wiener periodogram and Theorem 3 to sift out the data from a given fixed generator from the differenced data from two or more different generators (see Steps (6.)–(11.)).

The equidistribution of ${\Phi }_{\rho }\left(s_l\right)$ for $a.e.\rho$ leads to a Weyl flat range for S for

$$\rho \notin \{{\tau }_i/n:n\in \mathbb{N}\}.$$

Given that we have to produce an answer in finite time, and therefore, have to terminate, we pre-set a noise floor ${\eta }_0$ for the MEA component and a degree of accuracy parameter ${\mathcal{E}}_{\eta }$, which sets an iterative convergence baseline. The convergence rate of the algorithm is related to ${\eta }_0$ and ${\mathcal{E}}_{\eta }$. At each step, the data is sorted in descending order

$$S_{iter}={\left\{s_l\right\}}_{l=1}^N=\{s_1,\dots ,s_N\}.$$

We phase wrap the data by computing modulus of the spectrum, i.e., compute

$$|{\mathrm{Spec}}_{iter}\left(\tau \right)|=\left|{\displaystyle \sum _{l=1}^N}{e}^{(2\pi i{s}_l/\tau )}\right|.$$

The values of $|{\mathrm{Spec}}_{iter}\left(\tau \right)|$ will have peaks at the periods ${\tau }_i$ and their harmonics $\left({\tau }_i\right)/k$. Recall, letting $\rho \in [T_L,T_U]$, we define

$${\Phi }_{\rho }\left(s_l\right)=〈{\displaystyle \frac{s_l}{\rho }}〉={\displaystyle \frac{s_l}{\rho }}-⌊{\displaystyle \frac{s_l}{\rho }}⌋,$$

a function with range $[0,1)$. For $a.e.$ values of $\rho$, ${\Phi }_{\rho }\left(s_l\right)$ is essentially uniformly distributed in the sense of Weyl. Moreover, the set of $\rho$’s for which this is not true is rational multiples of $\left\{{\tau }_i\right\}$. The values at which ${\Phi }_{\rho }\left(s_l\right)=0$ are almost surely $\rho \in \{{\tau }_i/n:n\in \mathbb{N}\}$, which is a set of measure zero. This is why we get peaks at $\{{\tau }_i/n\}$.

Given $|{\mathrm{Spec}}_{iter}\left(\tau \right)|$, choose the rightmost peak. Label it as ${\tau }_{iter}$. The parameter ${\mathcal{E}}_{\eta }$ is used as a difference between iterations. If

$$|{\mathrm{Spec}}_{iter}\left({\tau }_{iter}\right)|>{\mathcal{E}}_{\eta },$$

(64)

declare $\widehat{{\tau }_i}={\tau }_{iter}$.

$$\underline{\mathbf{\text{The}} \mathbf{\text{EQUIMEA}} \mathbf{\text{Algorithm}}\mathbf{—}\mathbf{\text{Multiple}} \mathbf{\text{Periods}}}$$

Initialize: Sort the elements of S in descending order. Import ${\eta }_0$ [Eliminate $\phi$’s and initial noise.] Form the new set with elements $(s_l-s_{l+1})$. If $0\le s_j\le {\eta }_0$, then $S⟵S\setminus \left\{s_j\right\}$. Set $\mathtt{iter}=\mathtt{1}$, $i=1$, and compute ${\mathcal{E}}_{\eta }$.

(1.) 

[Adjoin 0.] $S_{iter}⟵S\cup \left\{0\right\}$.

(2.) 

[Sort.] Sort the elements of $S_{iter}$ in descending order.

(3.) 

[Compute all differences. This backfills the set S.] Set $S_{iter}=\bigcup (s_j-s_k)$ with $s_j>s_k$.

(4.) 

[Eliminate noise.] If $0\le s_j\le {\eta }_0$, then $S⟵S\setminus \left\{s_j\right\}$.

(5.) 

[Adjoin previous iteration.] Form $S_{iter}⟵S_{iter}\cup S_{iter-1}$, sort and reindex.

(6.) 

[Compute spectrum.] Compute $|{\mathrm{Spec}}_{iter}\left(\tau \right)|=|{\sum }_{l=1}^N{e}^{(2\pi i{s}_l/\tau )}|.$

(7.) 

[Threshold.] Choose the rightmost peak. Label it as ${\tau }_{iter}$.

(8.) 

[Test.] If $|{\mathrm{Spec}}_{iter}\left({\tau }_{iter}\right)|>{\mathcal{E}}_{\eta }$ declare $\widehat{{\tau }_i}={\tau }_{iter}$. If not, $\mathtt{iter}⟵(\mathtt{iter}+\mathtt{1})$. Go to (1.).

(9.) 

[Remove $\widehat{{\tau }_i}$ and harmonics.] Given $\widehat{{\tau }_i}$, remove it and its harmonics, i.e., $|{\mathrm{Spec}}_{iter}\left(\tau \right)|$ for $\widehat{{\tau }_i}/m$, $m\in \mathbb{N}$. Label as ${\mathrm{Notch}}_{iter}\left(\tau \right)$.

(10.) 

[Recompute frequency notched spectrum.] Compute $|{\mathrm{Spec}}_{iter}\left(\tau \right)-{\mathrm{Notch}}_{iter}\left(\tau \right)|.$

(11.) 

[Threshold.] If $|{\mathrm{Spec}}_{iter}\left(\tau \right)-{\mathrm{Notch}}_{iter}\left(\tau \right)|<{\mathcal{E}}_{\eta }$ algorithm terminates. Else, let $i⟵i+1$. Go to step (7.).

We then have to deinterleave the data. We use a standard discrete matched filtering algorithm, correlating a known delayed signal (a “template”) with an unknown signal to detect the presence of the template in the unknown signal. Here, our known signal has the form

$$\sum _{k\in \mathbb{Z}}\delta (t-k{\tau }_i),$$

(65)

a pulse train of period ${\tau }_i$ with no missing observations, and our unknown signal consists of those elements of the original data S generated by all of the underlying periods $\Gamma =\left\{{\tau }_i\right\}$.

Given the original data and the set of generating periods $\{{\tau }_1,\dots ,{\tau }_n\}$, convolve the data with

$$\sum _{k\in \mathbb{Z}}\delta (t-k{\tau }_1).$$

This convolution will identify the elements in the original data set S that are generated by the generating period ${\tau }_1$. Call these elements $S_{{\tau }_1}$.

Let $S_2=S\setminus S_{{\tau }_1}$. Convolve $S_2$ with

$$\sum _{k\in \mathbb{Z}}\delta (t-k{\tau }_2).$$

This second convolution will identify the elements in the data set $S_2$ that are generated by the generating period ${\tau }_2$. Call these elements $S_{{\tau }_2}$. Let $S_3=S\setminus S_{{\tau }_2}$. Repeat the process for ${\tau }_3$ up to ${\tau }_n$.

This process deinterleaves the data into components generated by individual generators ${\tau }_i$. Further analysis can be carried out on these individual components, e.g., using the analysis outline in Section 2.5.

The Parameter ${\mathcal{E}}_{\eta }$

The parameter ${\mathcal{E}}_{\eta }$ sets an iterative convergence baseline. The convergence rate of the algorithm is related to ${\eta }_0$ and ${\mathcal{E}}_{\eta }$.

Remark 9. 

If we had no missing observations and minimal noise, we could proceed as follows. Form the pulse train ${\sum }_{l=1}^N\delta (\tau -s_l)$, and then compute the data correlation

$$\mathcal{C}\left(\tau \right)={\displaystyle \sum _{l=1}^N}{\displaystyle \sum _{m=1}^l}\delta (\tau -s_l+s_m).$$

Phase wrap the data by computing modulus of the spectrum, i.e., computing

$$|{\mathcal{S}}_{iter}\left(\tau \right)|=\left|{\displaystyle \sum _{l=1}^N}{\displaystyle \sum _{m=1}^n}\delta (\tau -s_l+s_m)e^{(2\pi i({\displaystyle \frac{s_l}{(s_l-s_m)}}))}\right|.$$

The values of $|{\mathcal{S}}_{iter}\left(\tau \right)|$ will have peaks at the periods ${\tau }_j$. This relies on the following fact. If ω is an nth root of unity not equal to 1, then $1+\omega +\dots +{\omega }^{n-1}=0$.

However, given that we have (possibly very many) missing observations in our data, computing $|{\mathcal{S}}_{iter}\left(\tau \right)|$ will not produce clearly defined underlying periods. This is why we backfill to reinforce the data so that $|{\mathcal{S}}_{iter}\left(\tau \right)|$ reveals the underlying periods (see Step 5 in the EQUIMEA Algorithm (Section 2.1)).

The parameter ${\mathcal{E}}_{\eta }$ is computed from the initialized data. (A variation, with analysis, for setting ${\mathcal{E}}_{\eta }$ in certain cases is given in [39]. Their model, however, assumes no missing observations.) It is a type of a “spectral envelop,” but only requires the data from the first iteration $S_1={\left\{s_l\right\}}_{l=1}^N$. It is computed in the initialization step. We have assumed a priori that the periods $\left\{{\tau }_i\right\}$ lie in a range $[T_L,T_U)$. Let $T=T_U-T_L$. Given N data points, segment the data into N bins of width $T/N$. For each of these bins, we can compute the correlation and the spectrum of the data elements in the $K^{\mathrm{th}}$ bin. Let this data be represented as $S_K=\{s_1,s_2,\dots ,S_{N_K}\}.$ From the pulse train ${\sum }_{l=1}^{N_K}\delta (\tau -s_l),$ compute the data correlation

$${\mathcal{C}}_K\left(\tau \right)=\left|{\displaystyle \sum _{l=1}^{N_K}}{\displaystyle \sum _{m=1}^l}\delta (\tau -s_l+s_m)\right|$$

and the modulus of the spectrum, i.e., compute

$${\mathcal{S}}_K\left(\tau \right)=\left|{\displaystyle \sum _{l=1}^{N_K}}{\displaystyle \sum _{m=1}^l}{e}^{(2\pi i({\displaystyle \frac{s_l}{(s_l-s_m)}}))}\right|.$$

for $s_l$ in the $K^{\mathrm{th}}$ bin. Then

$${\mathcal{E}}_{\eta }=max\{{\mathcal{C}}_K,{\mathcal{S}}_K\}.$$

Given the rightmost peak, which is labeled as ${\tau }_{iter}$, if

$$|{\mathrm{Spec}}_{iter}\left({\tau }_{iter}\right)|>{\mathcal{E}}_{\eta },$$

(66)

declare $\widehat{{\tau }_i}={\tau }_{iter}$.

4. EQUIMEA Simulation Results

We now present the analysis of three data sets. The first had a single periodic generator, the second, two generators, and the third, three generators. All of these were sparse data sets, with ≈90% missing data elements. We used the EQUIMEA algorithm to isolate the periodicities in the data. Once the periods were isolated, we used convolution-matched filtering algorithms to deinterleave the data sets.

The data set in Figure 1 is generated by $\tau =1$, with 90% of the information randomly removed, and 10% jitter noise. The data set had 1000 total points. (For each original data set, we show a snapshot of the data in the interval $[0,100]$ and the deinterleaved data in $[0,200]$). Figure 2 shows the graph of $|{\mathrm{Spec}}_{iter}|$ after three iterations of the EQUIMEA. The value $\widehat{\tau }$ is the rightmost peak. Recall that, given a one period data set S, the period of that data set is the unique maximum value of $\tau$ such that the $k_j$ are all integers. Note that for any period ${\tau }_i$ that fits the form of S, the numbers ${\displaystyle \frac{{\tau }_i}{2}},{\displaystyle \frac{{\tau }_i}{3}},\dots$ (the harmonics of τ) are also possible values. Thus, what one sees in all of the spectral outputs are peaks at the values ${\tau }_i,{\displaystyle \frac{{\tau }_i}{2}},{\displaystyle \frac{{\tau }_i}{3}},\dots$. This explains the need for the following steps in the EQUIMEA algorithm:

(7.) 

[Threshold.] Choose the rightmost peak. Label it as ${\tau }_{iter}$.

(8.) 

[Test.] If $|{\mathrm{Spec}}_{iter}\left({\tau }_{iter}\right)|>{\mathcal{E}}_{\eta }$ declare $\widehat{{\tau }_i}={\tau }_{iter}$.

If not, $\mathtt{iter}⟵(\mathtt{iter}+\mathtt{1})$. Go to (1.).

(9.) 

[Remove ${\tau }_i$ and harmonics.] Given ${\tau }_i$, remove it and its harmonics $|{\mathrm{Spec}}_{iter}\left(\tau \right)|$ for $\widehat{{\tau }_i}/m$, $m\in \mathbb{N}$. Label as ${\mathrm{Notch}}_{iter}\left(\tau \right)$.

(10.) 

[Recompute frequency notched spectrum.] Compute $|{\mathrm{Spec}}_{iter}\left(\tau \right)-{\mathrm{Notch}}_{iter}\left(\tau \right)|.$

Also note that the EQUIMEA algorithm can extract the period from extremely sparse data ($\approx 90$% of the information randomly removed, and 10% jitter noise—Figure 1), and tell us, after the “frequency notching step,” that there is only one period in the data set (Figure 2).

Figure 1. Sparse data set containing one underlying period.

Figure 1. Sparse data set containing one underlying period.

Figure 2. One period EQUIter3 spectrum.

Figure 2. One period EQUIter3 spectrum.

The data in Figure 3 had two underlying periods equaling 1 and $\varphi =(1+\sqrt{5})/2$, with 90% of the information randomly removed and 10% jitter noise. Figure 4 shows $|{\mathrm{Spec}}_{iter}|$ after two iterations. By proceeding right to left, one can easily see the two underlying periods—${\displaystyle \frac{1+\sqrt{5}}{2}}$ and 1. Each will reinforce the data elements from the specific generator, allowing the elements to be extracted. The algorithm backfilled the multiples of 1 and ${\displaystyle \frac{1+\sqrt{5}}{2}}$. The facts that there was a sufficient amount of data, in addition to the ergodicity of the periods relative to each other both played significant roles in making the periods stand out against the Weyl flat data.

The strengths of the signals $|{\mathrm{Spec}}_{iter}|$ are reliant on the fact that the generators are incommensurate, and so the differences of data elements $({\phi }_i+k_{ij}{\tau }_i+{\eta }_{ij})-({\phi }_{i^{\prime }}+k_{i^{\prime }{j}^{\prime }}{\tau }_{i^{\prime }}+{\eta }_{i^{\prime }{j}^{\prime }}),$ for $i\ne i^{\prime }$, are incommensurate to ${\tau }_i$ and ${\tau }_{i^{\prime }}$. These data elements in $|{\mathrm{Spec}}_{iter}|$ become Weyl flat. But this is a function of the degree of the accuracy of the computation, the ergodicity of the periods relative to each other, and, if applicable, to Diophantine approximations of the periods. We again note that very deep work of Katok, Stepin, Margulis, et al. [32,33] addresses this.

Figure 4 shows $|{\mathrm{Spec}}_{iter}|$ after two iterations. The two underlying periods—${\displaystyle \frac{1+\sqrt{5}}{2}}$ and 1—are clear as the peaks furthest to the right. Deinterleaving was then done by a matched filtering technique by convolving the original data set with the individual pulse trains

$$\sum _{k\in \mathbb{Z}}\delta (t-k(1+\sqrt{5})/2)\mathrm{and}\sum _{k\in \mathbb{Z}}\delta (t-k).$$

Each pulse train reinforces the data elements from the specific generator, allowing the elements to be extracted. Figure 5 shows the data deinterleaved, demarked by color. The red elements were generated by 1, the green by ${\displaystyle \frac{1+\sqrt{5}}{2}}$.

The data in Figure 6 had three underlying periods equaling ${\tau }_1=1$, ${\tau }_2=\varphi =(1+\sqrt{5})/2$, and ${\tau }_3=\sqrt{7}$. The data set had 90% of the information randomly removed and 10% jitter noise. The original data is the union of three data sets, totaling 3000 data points. Figure 7 shows $|{\mathrm{Spec}}_{iter}|$ after two iterations of the EQUIMEA. By proceeding right to left, one can easily see the three underlying periods—$\sqrt{7}$, ${\displaystyle \frac{1+\sqrt{5}}{2}}$, and 1. (Step 9 (“Frequency notching”) eliminates the first harmonic peak of $\sqrt{7}$, located at $\sqrt{7}/2$, and so the peak at 1 stands out.) Deinterleaving was then done by a matched filtering technique by convolving the original data set with the individual pulse trains

$$\sum _{k\in \mathbb{Z}}\delta (t-k\sqrt{7}),\sum _{k\in \mathbb{Z}}\delta (t-k(1+\sqrt{5})/2),\mathrm{and}\sum _{k\in \mathbb{Z}}\delta (t-k).$$

Again, each pulse train reinforces the data elements from the specific generator, allowing the elements to be extracted. Figure 8 shows the data interleaved, demarked by color. The red elements were generated by 1, the green by ${\displaystyle \frac{1+\sqrt{5}}{2}}$, and cyan by $\sqrt{7}$.

The EQUIMEA is reliant on the “incommensurate nature” of the periods to each other. This manifests itself in the difference of data elements

$$({\phi }_i+k_{ij}{\tau }_i+{\eta }_{ij})-({\phi }_{i^{\prime }}+k_{i^{\prime }{j}^{\prime }}{\tau }_{i^{\prime }}+{\eta }_{i^{\prime }{j}^{\prime }}).$$

for $i\ne i^{\prime }$. The elements are almost surely incommensurate to ${\tau }_i$ and ${\tau }_{i^{\prime }}$, i.e., not equal to a rational multiple of either number.

We note that if two underlying processes have commensurate (or even equal) periods, the EQUIMEA will first capture the largest of the periods (or the period). It is highly probable that the two or more events will likely not have the same phase. When this happens, the original data can then be separated by the deinterleaving process.

The “Spectrum” of Point Processes Tools

We close this section by addressing the role the number theoretic methods can play in the “spectrum” of the tools used to analyze point processes. The focus of this paper (the MEA/EQUIMEA approach) is only one component of an extensive toolbox to attack point process problems. The choice of the best tool to use comes from a preliminary analysis of the data, including characteristics of the data, how it was generated, and a sense of the SNR and the sparsity of the data elements.

Least squares, periodograms, and the MEA/EQUIMEA are all useful in the analysis of general point process data. The sparsity, as determined by the $k_j$’s, provides guidance toward the best procedure for analyzing this data. Given a sequence of consecutive $k_j$’s or even an explicit knowledge of the $k_j$’s, and assuming white Gaussian noise, we can get a maximum likelihood estimate using least squares. Periodograms work with some missing observations, but when the percentage of missing observations is too large (>50–75%), they begin to break down. Number theoretic methods can work with very sparse data sets (>90% missing observations). There is, however, a trade-off. Given 10 data elements with a high signal-to-noise ratio and no outliers, the number theoretic methods can produce reliable estimates. But these too can break down. Given an extremely noisy environment and a large number of outliers, the number theoretic methods require considerably larger data sets to get reliable estimates. If the period $\tau$ is much lower than the noise, the MEA will create false outliers, producing unreliable answers.

5. Theory vs. Computation

We finish the paper with a discussion of computability. The following are well-known results on Fibonacci numbers and the Golden Mean $\varphi$.

Lemma 3 

(Fibonacci and $\varphi$). Let ${\mathcal{F}}_1={\mathcal{F}}_2=1$, ${\mathcal{F}}_{n+1}={\mathcal{F}}_n+{\mathcal{F}}_{n-1}$, the Fibonacci numbers, and $\varphi =(1+\sqrt{5})/2$. Then $\varphi =[1,1,1,1,\dots ]$, i.e.,

$$\varphi =1+{\displaystyle \frac{1}{1+\frac{1}{1+\frac{1}{1+\cdots }}}}$$

and

$${\displaystyle \underset{n\to \infty }{lim}{\mathcal{F}}_{n+1}/{\mathcal{F}}_n=\varphi .}$$

Thus, given any $ϵ>0$, there exists infinitely many Fibonacci numbers such that

$$\left|\right({\mathcal{F}}_{n+1}/{\mathcal{F}}_n)-\varphi |<ϵ.$$

Ergodicity plays a key role in the efficiency of the EQUIMEA. We set up computational experiments using two numbers—

$$\varphi ={\displaystyle \frac{1+\sqrt{5}}{2}}\mathrm{and}\mathbf{L}={\displaystyle \sum _{k=1}^{\infty }}1/{10}^{k!}.$$

Here, $\varphi$, the Golden Mean, is poorly approximated by rationals, whereas L is the Liouville constant, a transcendental well approximated by rationals. We first note that the EQUIMEA algorithm on data sets with two generators—1 and L—will not work. The number L is irrational, but it is also well approximated by rationals. In fact, L $=0.110010000000000000000010\dots ,$ and the gaps between non-zero digits become so large that the EQUIMEA numerically views it as a rational. Thus, even though L is irrational, and therefore Weyl’s Theorem holds, the computations of the EQUIMEA can not separate the periods. The key, as we will show, is ergodicity.

Definition 5. 

A real number β is a Liouville number, denoted by $\beta \in \mathcal{L}$, if β is irrational and if for every integer $m\ge 2$, there exist integers $p,q$ with $q\ge 2$ such that

$$\left|\beta -{\displaystyle \frac{p}{q}}\right|<{\displaystyle \frac{1}{{\left|q\right|}^m}}.$$

(67)

Liouville developed the Liouville numbers $\mathcal{L}$ as a special class of transcendentals. The complement of $\mathcal{L}$ includes poorly approximated by rationals. The Golden Mean $\varphi =(1+\sqrt{5})/2$ is the least rapidly approximated, and this is seen in its continuant $[1;\overline{1}]$. In contrast, the continuant of L is $[0;9,11,99,\dots ]$, with gaps between increasingly large sequences of 9’s. In terms of their approximation by rationals, $\varphi$ and L are quite different. $|{\mathrm{Spec}}_{iter}|$ will become Weyl flat very quickly for the two periods 1 and $\varphi$ (see Figure 3). The ergodicity of the periods relative to each other plays a role in making the periods stand out against the “flat” data. In this sense, although Weyl’s theorem holds for all irrationals, its “numerical erogdicity” is a function of the continuants, and where the behaviour of these continuants lies, with $\varphi$ and L being at opposite extremes. This is evident in the following numerical computation.

We plot $\exp \left(2\pi in{\tau }_j\right)$ for

$${\tau }_1=\varphi ={\displaystyle \frac{1+\sqrt{5}}{2}}\mathrm{and}{\tau }_2=\mathbf{L}=\sum 1/{10}^{k!}.$$

In Figure 9, we see the ergodicity of $\varphi$, essentially “filling” the circle with only 300 points. In contrast, Figure 10 shows that $\mathcal{L}$ at first seems to follow the distribution of $\tau =11/100$, and, even after 3000 points, has noticeable missing regions. But $\mathcal{L}$ is irrational, and therefore, by Weyl’s Theorem, the points will, in theory, eventually fill in. However, numerically, L$=0.1100010000000000000000010\dots ,$ and the gaps between non-zero digits in L becomes so large that the missing regions will always be present in any numerical experiment.

• However, encountering an element of $\mathcal{L}$ is unlikely.

If $\sigma$ is a positive real number, and $X\subset \mathbb{R}$, then we say that X has σ-dimensional Hausdorff measure zero if for every $ϵ>0$, there is a sequence of intervals $I_n$ such that

$$X\subset {\displaystyle \bigcup _{n=1}^{\infty }}{I}_n,$$

$$\mathrm{length}\left(I_n\right)<ϵ\mathrm{for}\mathrm{all}n,\mathrm{and}$$

$${\displaystyle \sum _{n=1}^{\infty }}{\left(\mathrm{length}\left(I_n\right)\right)}^{\sigma }<ϵ.$$

If $\sigma =1$, X has Lebesgue measure zero. If $0<\sigma <1$, the condition is stronger. For example, if $\mathcal{C}$ denotes the Cantor middle-thirds set, then $\mathcal{C}$ has Lebesgue measure zero, but for any $\sigma <\log \left(2\right)/\log \left(3\right)$, does not have $\sigma$-dimensional Hausdorff measure zero.

Theorem 4. 

The class of Liouville numbers $\mathcal{L}$ has Lebesgue measure zero.

Remark 10. 

In fact, $\mathcal{L}$ has σ-dimensional Hausdorff measure zero for all $\sigma >0$.

Proof. 

Let $\beta \in \mathcal{L}$. Then, for all $n\in \mathbb{N}$ there exists $p,q$ such that $q>2$, $p/q\in \mathbb{Q}$, and

$$\left|\beta -{\displaystyle \frac{p}{q}}\right|<{\displaystyle \frac{1}{{\left|q\right|}^n}}.$$

(68)

Now let

$${\mathbb{Q}}_n={\displaystyle \bigcup _{q=2}^{\infty }}{\displaystyle \bigcup _{p=-\infty }^{\infty }}\left({\displaystyle \frac{p}{q}}-{\displaystyle \frac{1}{q^n}},{\displaystyle \frac{p}{q}}+{\displaystyle \frac{1}{q^n}}\right).$$

(69)

Then, ${\mathbb{Q}}_n$ is a countable union of open intervals. Moreover, ${\mathbb{Q}}_n$ includes every number of the form $p/q$, for $q\ge 2$. We claim that $\beta \in \mathcal{L}$ if and only if

$$\beta \in (\mathbb{R}\setminus \mathbb{Q})\cap \left({\displaystyle \bigcap _{n=1}^{\infty }}{\mathbb{Q}}_n\right).$$

(70)

But since

$$\left|\beta -{\displaystyle \frac{p}{q}}\right|<{\displaystyle \frac{1}{{\left|q\right|}^n}}$$

for all n and all $q\ge 2$,

$${\displaystyle \frac{-1}{{\left|q\right|}^n}}<\left(\beta -{\displaystyle \frac{p}{q}}\right)<{\displaystyle \frac{1}{{\left|q\right|}^n}},$$

and so

$${\displaystyle \frac{p}{q}}-{\displaystyle \frac{1}{{\left|q\right|}^n}}<\beta <{\displaystyle \frac{p}{q}}+{\displaystyle \frac{1}{{\left|q\right|}^n}}.$$

This proves the claim. Thus, $\mathcal{L}\subset {\mathbb{Q}}_n$ for every n. Now, let

$${\mathbb{Q}}_{n,q}={\displaystyle \bigcup _{p=-\infty }^{\infty }}\left({\displaystyle \frac{p}{q}}-{\displaystyle \frac{1}{q^n}},{\displaystyle \frac{p}{q}}+{\displaystyle \frac{1}{q^n}}\right),$$

(71)

for $q=2,3,\dots$. For any $m,n\in \mathbb{N}$,

$$\begin{array}{cc} & {\displaystyle \mathcal{L}\cap (-m,m)\subset {\mathbb{Q}}_n\cap (-m,m)}\hfill \\ =& {\displaystyle \bigcup _{q=2}^{\infty }}\left({\mathbb{Q}}_{n,q}\cap (-m,m)\right)\subset {\displaystyle \bigcup _{q=2}^{\infty }}{\displaystyle \bigcup _{p=-mq}^{mq}}\left({\displaystyle \frac{p}{q}}-{\displaystyle \frac{1}{q^n}},{\displaystyle \frac{p}{q}}+{\displaystyle \frac{1}{q^n}}\right).\hfill \end{array}$$

(72)

Let $ϵ>0$ be given. For each $m\in \mathbb{N}$ we want to find a sequence of intervals $I_n$ such that

$$\begin{array}{c}\mathcal{L}\cap (-m,m)\subset {\displaystyle \bigcup _{n=1}^{\infty }}{I}_n,\hfill \\ \mathrm{length}\left(I_n\right)<ϵ\mathrm{for}\mathrm{all}n,\mathrm{and}\hfill \\ {\displaystyle \sum _{n=1}^{\infty }}{\left(\mathrm{length}\left(I_n\right)\right)}^{\sigma }<ϵ.\hfill \end{array}$$

(73)

Choose n so that

$${\displaystyle \frac{1}{2^{n-1}}}<ϵ,$$

$$n\sigma >2,\mathrm{and}\left({\displaystyle \frac{(2m+1)2^{\sigma }}{(n\sigma -2)}}\right)<ϵ.$$

(74)

For n which satisfy these conditions,

$$\mathrm{length}\left({\displaystyle \frac{p}{q}}-{\displaystyle \frac{1}{q^n}},{\displaystyle \frac{p}{q}}+{\displaystyle \frac{1}{q^n}}\right)={\displaystyle \frac{2}{q^n}}\le {\displaystyle \frac{2}{2^n}}<ϵ.$$

(75)

We also have

$$\begin{array}{cc} & {\displaystyle \sum _{q=2}^{\infty }}{\displaystyle \sum _{p=-mq}^{p=mq}}{\left({\displaystyle \frac{2}{q^n}}\right)}^{\sigma }={\displaystyle \sum _{q=2}^{\infty }}{\displaystyle \frac{(2mq+1)2^{\sigma }}{q^{n\sigma }}}\le {\displaystyle \sum _{q=2}^{\infty }}{\displaystyle \frac{(2mq+q)2^{\sigma }}{q^{n\sigma }}}\hfill \\ =& (2m+1)2^{\sigma }{\displaystyle \sum _{q=2}^{\infty }}{\displaystyle \frac{1}{q^{(n\sigma -1)}}}\le (2m+1)2^{\sigma }{\int }_1^{\infty }{\displaystyle \frac{dt}{t^{(n\sigma -1)}}}\hfill \\ =& \left({\displaystyle \frac{(2m+1)2^{\sigma }}{(n\sigma -2)}}\right)<ϵ.\hfill \end{array}$$

(76)

This completes the proof. □

We computed the gaps between the dots.

• For 300 rotations of $\varphi$, the biggest gap between angles is 0.0316 (about 1.8 degrees).
• For 3000 rotations of Liouville’s number, the biggest gap between angles is 0.0452 (about 2.6 degrees).

The latter is about 1.4 times the former, even though it has 10 times as many dots.