The Rise of the Brown–Twiss Effect

Abstract

Despite the simplicity of flux collecting hardware, robustness to misalignments, and immunity to seeing conditions, Intensity Correlation Imaging arrays using the Brown–Twiss effect to determine two-dimensional images have been burdened with very long integration times. The root cause is that the essential phase retrieval algorithms must use image domain constraints, and the traditional signal-to-noise calculations do not account for these. Thus, the conventional formulations are not efficient estimators. Recently, the long integration times have been emphatically removed by a sequence of papers. This paper is a review of the previous theoretical work that removes the long integration times, making the Intensity Correlation Imaging a practical and inexpensive method for high-resolution astronomy.

1. Introduction

The first experiments on imaging techniques based upon the correlation of intensity fluctuations measured at two or more spatial–temporal points for thermal sources of light were carried out by R. Hanbury Brown and R. Q. Twiss (HBT) [1,2,3,4]. From the outset, it was clear that this first exploration in quantum optics had many advantages over conventional telescopes and amplitude interferometers. Among these advantages are the followsing:

▪

HBT does not require the propagation of collected light beams to combiner units as in existing amplitude interferometers. Routing optics and beam combiners are eliminated.

▪

HBT does not require nanometer-level control of nor nanometer knowledge of the relative positions of the optical components.

▪

The signal-to-noise ratio (SNR) does not suffer from beam splitting and the throughput losses that plague amplitude interferometry.

▪

HBT does not require expensive diffraction-limited optics. Photo detective hardware consists of simple, flux collecting apertures.

▪

HBT is insensitive to atmospheric turbulence and therefore can be used by ground-based as well as space-based facilities.

▪

Above all, each aperture transmits its own intensity fluctuation signal. All pairs of these signals are correlated via a separate computational facility. Thus each pair of apertures are electronically coupled so that the relative distance vectors between apertures (the baseline vectors) can be arbitrarily large, allowing for the possibility of indefinitely fine image resolution.

Unfortunately, for HBT correlators, the measurement signal is proportional to the square of the average light intensity. Most astronomical light sources are not bright enough to determine stellar diameters with reasonable integration times (hours or minutes rather than millions of hours). Until very recently, integration time has been the major obstacle for measurements of stellar diameters. A second difficulty is that HBT has been concerned with the calculation of stellar diameters modeled as radially symmetric intensity distributions that require only the measurement of coherence magnitudes. For much broader astronomical investigations, an algorithm capable of two-dimensional imaging is needed. This requires both coherence magnitude and coherence phase retrieval. Finally, in the interests of astronomy, we concentrate on imaging an illuminated body against a black sky where the positions of the black pixels are unknown but are determined by ICI.

The present discussion is a survey of a sequence of four papers [5,6,7,8] that developed an algorithm for two-dimensional imaging that enormously reduces the integration times for the HBT effect while retaining the advantages listed above. First of all, Reference [5] developed the noise reducing phase retrieval (NRPR) algorithm. This led to three papers [6,7,8] produced by the author as an independent researcher, and these three most recent papers were the most instrumental in solving the curse of the weak signal. The four articles were arranged in both topical and chronological order with rigorous technical detail. While the salient features of the mathematical development are herein presented, this paper encourages its users to appreciate the rather simple process for the determination of high-resolution images from the extremely noisy database. We emphasize that the HBT technique has been called “Intensity Correlation Interferometry”. However, this significant step forward merits the name “Intensity Correlation Imaging” (which we designate as ICI hereafter).

The plan of the present development is as follows. In the next section, we explain the coherence magnitude measurement process and a reformulation of the HBT SNR (signal-to-noise ratio) that reflects the fact that the conditional probabilities arising from the image domain constraints (needed in the Cramér–Rao bound) must be accounted for in any estimate of measurement accuracy as a function of integration time. In other words, the conventional SNR treatments [1,2,3,4] are inappropriate in the context of two-dimensional image reconstruction. Evidence of this appeared in earlier studies, which showed that conventional estimates of integration time for many imaging examples were grossly conservative. This section explains why the ICI data collection process must utilize coherence magnitudes that are strictly nonnegative in order to correct the HBT SNR.

Next, in Section 3, we offer an overview of the noise reducing phase retrieval (NRPR) algorithm [5]. This section postulates a complete knowledge of the positions of the black pixels, wherein an illuminated, square foreground of a specified size is centered amid the constrained black pixels of the background. This is termed the “Box”. Using NRPR to compute a multitude of “Box” sizes, we present computational studies that examine the correct integration time under the assumption that integration times are the same for all aperture pairs (i.e., all baselines in the u-v plane). The results show an enormously more precise estimate for integration time as a function of image size and accuracy.

Section 4 describes the technique wherein the black pixels in the background having specified the “Box” size are able to determine the zero-noise image. The process embeds the NRPR algorithm (equipped with the correct integration times) within a stochastic search algorithm [8]. By varying the box sizes on each NRPR run, the occurrence of two completely correlated images (aside from 180-degree rotations and translations) indicates the correct zero-noise image. The correct identification is attained via very few iterations.

Section 5 discusses theorems that lead to an even faster identification of the correct, zero-noise image. Then, in the Section 7, we discuss further possibilities for both the advancement of the ICI algorithm within the photonic community and its application to advanced image resolution.

2. ICI Data Collection Process

For simplicity, we postulate a set of flux collecting apertures forming a square grid on the u-v plane. Furthermore, the parameters of all apertures, including the integration times (denoted by $\Delta T$), are identical. For a given pair of apertures, the fundamental data collected by Brown and Twiss to obtain an estimate of the coherence magnitude are a running time average of the intensity fluctuation cross-correlation as measured by each pair of intensity sensors. We designate $u$ as the baseline vector in the u-v plane associated with the two apertures. Here, we reconsider the statistics of intensity fluctuation cross-correlation that prevail when the object of study has very low spectral radiance. For the present discussion, we assume that (1) the averaging time, $\Delta T$, is much larger than the detector response time, $T_d$; (2) the field is quasi-monochromatic, i.e., that it is confined to a narrow spectral band, $\Delta {\nu }_c$, centered at frequency $\overline{\nu }$ such that $\Delta {\nu }_c\ll \overline{\nu }$; (3) the light is approximately cross-spectrally pure, i.e., the normalized mutual coherence takes the form $\gamma \left(r_1,t_1;r_2,t_2\right)=\gamma \left(r_1,r_2,0\right)\gamma \left(t_2-t_1\right)$, which implies that the coherence time of the light satisfies $T_c={{\displaystyle {\int }_{-\infty }^{\infty }\left|\gamma \left(\tau \right)\right|}}^{2}d\tau =1/\Delta {\nu }_c$ ($\Delta {\nu }_c$ denoting the optical bandwidth); (4) the slow detector assumes that $T_d\gg T_c$; and (5) the thermal light radiated by the targets to be imaged has very small total photodetection events per second per Hertz (a restriction almost universally valid). For simplicity of exposition, we also assume that the intensity and mutual coherence magnitudes are approximately constant over the spatial extent of each aperture pair. In other words, the partial coherence factor calculated in Reference [2] is approximately unity.

Let us introduce some further notation. The symbol $〈\dots 〉$ denotes the ensemble average, and ${〈\dots 〉}_{\Delta T}$ denotes the time average, defined as follows:

$${〈\dots 〉}_{\Delta T}={\displaystyle \frac{1}{\Delta T}}{\displaystyle {\int }_{t-\Delta T}^t\left(\dots \right) }d\xi$$

(1)

The prefixes “$\Delta$” and “${\Delta }_T$” indicate fluctuations about the ensemble average and about the time average, respectively. Finally, let the electronic outputs of the two detectors be denoted by $J_1\left(t,u\right) \mathrm{and} J_2\left(t,u\right)$. Then, the time-averaged intensity fluctuation cross-correlation (the principal datum of the measurement) is ${〈{\Delta }_{\Delta T}{J}_1\left(t,u\right){\Delta }_{\Delta T}{J}_2\left(t,u\right)〉}_{\Delta T}$.

The statistics of the time-averaged intensity fluctuation cross-correlation (the quantity ${〈{\Delta }_{\Delta T}{J}_1\left(t,u\right){\Delta }_{\Delta T}{J}_2\left(t,u\right)〉}_{\Delta T}$) has been treated in great detail in [1,2,3,4]. Accordingly, assumption (5), which holds for all but the brightest objects, leads to the conclusion that in the presence of conditions (1)–(4) and at any particular integration time where ${〈{\Delta }_{\Delta T}{J}_1\left(t,u\right){\Delta }_{\Delta T}{J}_2\left(t,u\right)〉}_{\Delta T}$ is a Gaussian random variable, the mean and standard deviation are as follows:

$$\begin{array}{c}{〈{\Delta }_{\Delta T}{J}_1\left(t,u\right){\Delta }_{\Delta T}{J}_2\left(t,u\right)〉}_{\Delta T}={\kappa }^2{\eta }^2{\displaystyle \frac{T_d}{T_c}}{\overline{\beta }}^2{\left|\gamma \right|}^2+\left[{\kappa }^2\eta {\displaystyle \frac{T_d}{T_c}}\overline{\beta }\right]\sqrt{{\displaystyle \frac{T_d}{\Delta T}}} N_u\left(0,1\right)\\ \\ \overline{\beta }=\mathrm{Number} \mathrm{density} \mathrm{per} \mathrm{mode} \cong AI{T}_c \\ A=\mathrm{Aperture} \mathrm{area}\\ I=\mathrm{Number} \mathrm{of} \mathrm{photon} \mathrm{arrivals} \mathrm{per} \mathrm{square} \mathrm{meter}, \mathrm{per} \mathrm{second}\\ T_c=\mathrm{Correlation} \mathrm{time} \mathrm{of} \mathrm{the} \mathrm{light}\\ T_d=\mathrm{Response} \mathrm{time} \mathrm{of} \mathrm{the} \mathrm{detector}\\ \kappa =\mathrm{Magnitude} \mathrm{of} \mathrm{detector} \mathrm{response} \mathrm{over} T_d\\ \eta =\mathrm{Photon} \mathrm{detection} \mathrm{efficiency}\\ \left|\gamma \left(u\right)\right|\triangleq \left|\gamma \left(r_1,r_2,0\right)\right|\\ =\mathrm{The} \mathrm{normalized} \mathrm{magnitude} \mathrm{of} \mathrm{optical} \mathrm{coherence}\end{array}$$

(2)

$N_u\left(0,1\right)$ is a zero mean, unit variance Gaussian random variable where $N_u\left(0,1\right)$ and $N_{u^{\prime }}\left(0,1\right)$ are statistically independent if $u\ne u^{\prime }$. Note that as the integration time approaches infinity, ${〈{\Delta }_{\Delta T}{J}_1\left(t,u\right){\Delta }_{\Delta T}{J}_2\left(t,u\right)〉}_{\Delta T}$ converges to a constant multiplier times the magnitude squared of the normalized degree of optical coherence, ${\left|\gamma \left(u\right)\right|}^2$. This is the primary discovery of Brown and Twiss for the case of thermal light.

At this point, we observe that the coefficient of the noise term can be determined at the outset by calibrating the detectors, using the aperture parameters, and measuring the intensity observed by the individual aperture sensors. In other words, the quantity $\left[{\kappa }^2\eta \overline{\beta }{T}_d/T_c\right]\sqrt{T_d/\Delta T}$ can be obtained before any intensity fluctuation cross-correlation measurements are attempted. Hence, it is convenient to divide ${〈{\Delta }_{\Delta T}{J}_1\left(t\right){\Delta }_{\Delta T}{J}_2\left(t\right)〉}_{\Delta T}$ by this factor, thereby normalizing the noise component to unity standard deviation. Therefore, we can write the equation as follows:

$$\begin{array}{c}S\left(u\right)={\displaystyle \frac{{〈{\Delta }_{\Delta T}{J}_1\left(t,u\right){\Delta }_{\Delta T}{J}_2\left(t,u\right)〉}_{\Delta T}}{\left[{\kappa }^2\eta \overline{\beta }{T}_d/T_c\right]\sqrt{T_d/\Delta T} }}=S_0\sqrt{\Delta T}{\left|\gamma \left(u\right)\right|}^2+N_u\left(0,1\right)\\ S_0=\eta AI{T}_c\sqrt{1/T_d}\end{array}$$

(3)

$S_0$ may be recognized as the zero baseline SNR for a unit integration time, which can be determined at the outset via measurements from the individual apertures and equipment calibrations. It is important to note that the statistical properties of $S\left(u\right)$ and $S_0$ are completely determined by the parameters $\left\{\kappa , \eta , T_d, T_c ,A ,I, \overline{\beta }=AI{T}_c\right\}$ aside from ${\left|\gamma \left(u\right)\right|}^2$ and $\sqrt{\Delta T}$. Any statistical data regarding supposedly specific radio astronomy, adaptive optics, or other noise distributions are carried within these parameters.

Clearly, the meaning of $S\left(u\right)$ is the same as the time-averaged cross-correlation, and they are interchangeable. However, $S\left(u\right)$ is more easily interpreted, as it is the “empirical SNR” that one can use to compare the signal component of the measurement to the noise component, whose standard deviation is always one. Hence, as the integration time increases, one can compare $S_0\sqrt{\Delta T}{\left|\gamma \left(u\right)\right|}^2$ to unity to ascertain if the desired SNR is reached with a 50% probability. $S\left(u\right)$ can be used to illustrate the process of experiment design and data gathering and demonstrates the essential differences between the measurement of stellar diameters versus the capture of two-dimensional images.

Let us discuss the initial experiment design as practiced by Brown and Twiss. Obviously, one cannot prolong the running time average to an impractical degree. At the outset, one must estimate the length of the final time average, denoted here by $\Delta T$ so that the final empirical SNR is roughly of the same magnitude as the desired SNR, SNR_d, allowing for an approximately unit standard deviation fluctuation. Based upon the above description of the estimation process, Equation (3) leads to the following specification of the limiting integration time, denoted by $\Delta T$:

$$\Delta T={\left({\displaystyle \frac{SN{R}_d}{S_0}}\right)}^2{\displaystyle \frac{1}{{\left|{\gamma }_{est}\right|}^4}}$$

(4)

where $S\left(u\right)$ has been replaced by ${SNR}_d$ in (3) to arrive at this expression and $\left|{\gamma }_{est}\right|$ is some estimate of the coherence magnitude. As expected, any reduction of the factor $S_0$ (traditionally characterized by the inefficiency factor) lengthens the necessary integration time so that accurate equipment calibration is essential. We also see that design of in-the-field measurements needs an a priori estimate of $\left|{\gamma }_{est}\right|$.

The process adopted by Brown and Twiss to measure stellar diameters involves measuring only the coherence magnitudes and curve fitting the results to a simple, radially symmetric model of the stellar intensity pattern. Because of the simplicity of the technique, one finds that reasonable accuracy for the Brown–Twiss calculation may be obtained using a $\left|{\gamma }_{est}\right|$ close to unity (note that the normalized coherence has the properties $\left|\gamma \left(u\right)\right|>0, {\max }_u\left|\gamma \left(u\right)\right| =\left|\gamma \left(u=0\right)\right|=1$). In fact, some equalities in Refs. [1,2,3,4] actually assign the factor $\left|\gamma \left(u\right)\right|$ to unity, although a more precise estimate is likely to be $\left|{\gamma }_{est}\right|\simeq 0.2\text{–}0.3$. This estimate of integration time is roughly correct for stellar interferometry and provides a good starting value to the needed integration time. Also, the quantity $S_0\sqrt{\Delta T}$ may be considered an efficient estimator of the coherence magnitude squared in that it is inherently positive and does not depend on the value of the quantity being estimated (namely the normalized magnitude of the optical coherence). In contrast, a fully two-dimensional image of a complex object, based on interferometric experience, requires at least $\left|{\gamma }_{est}\right|\simeq 0.01\text{–}0.001$. Thus, in this case, the method used by Brown and Twiss to determine a two-dimensional image is clearly inappropriate.

The inconvenient aspect of the raw, noisy data delivered by the empirical SNR is that it cannot be guaranteed to be nonnegative, and therefore, the noisy data cannot be considered an efficient estimator for the purpose of two-dimensional imaging. Further, the Cramér–Rao bound entails that the Fisher information include conditional probabilities that are necessary for phase retrieval algorithms that demand positivity. Indeed, $S\left(u\right)$ can assume negative values with high probability when $\Delta T$ is sufficiently small. Before proceeding further, we remedy this by formulating a nonnegative function of the measurements, without somehow ignoring negative-valued data. We take some pairs of independent measurements, say, the following:

$$\begin{array}{l} S^{\left(1\right)}\left(u\right)=S_0\sqrt{\Delta T} {\left|\gamma \left(u\right)\right|}^2+N_u^{\left(1\right)}\left(0,1\right),\\ S^{\left(2\right)}\left(u\right)=S_0\sqrt{\Delta T} {\left|\gamma \left(u\right)\right|}^2+N_u^{\left(2\right)}\left(0,1\right), \end{array}$$

(5)

then multiply one of them by the unit imaginary, add the results, evaluate the square root of the magnitude, and divide by $\sqrt{S_0\sqrt{2\Delta T}}$, which is known in advance, to obtain the following:

$$\hat{G}\left(u,\Delta T\left(u\right)\right)={\displaystyle \frac{1}{\sqrt{S_0\sqrt{2\Delta T\left(u\right)} }}}{\left|S^{\left(1\right)}\left(u\right)+i{S}^{\left(2\right)}\left(u\right)\right|}^{1/2}$$

(6)

where the nonnegative quantity, $\hat{G}\left(u,\Delta T\right)$, now takes the place of the coherence magnitude measurement. Substituting the expressions in (5) for $S^{\left(1\right)}\left(u\right) \mathrm{and} S^{\left(2\right)}\left(u\right)$, we have the following:

$$\hat{G}\left(u,\Delta T\right)=\left|\gamma \left(u\right)\right|{\left| 1+{\displaystyle \frac{1}{{\left|\gamma \left(u\right)\right|}^2{S}_0\sqrt{2\Delta T}}}{\displaystyle \frac{1}{1+i}}\left(N_u^{\left(1\right)}\left(0,1\right)+i{N}_u^{\left(2\right)}\left(0,1\right)\right)\right|}^{1/2}$$

(7)

However, ${\displaystyle \frac{1}{1+i}}\left(N_u^{\left(1\right)}\left(0,1\right)+i{N}_u^{\left(2\right)}\left(0,1\right)\right)$ is equivalent to $\left(N_{1u}\left(0,1\right)+i{N}_{2u}\left(0,1\right)\right)/\sqrt{2}$, where $N_{1u}\left(0,1\right)$ and $N_{2u}\left(0,1\right)$ are independent Gaussian random variables with zero mean and unit variance. Therefore, the following nonnegative quantity may be taken as a single measurement:

$$\hat{G}\left(u,\Delta T\left(u\right)\right)=\left|\gamma \left(u\right)\right|{\left|1+{\displaystyle \frac{\left(N_{1u}\left(0,1\right)+i{N}_{2u}\left(0,1\right)\right)}{S_0\sqrt{2\Delta T} {\left|\gamma \left(u\right)\right|}^2}}\right|}^{1/2}$$

(8)

The quantity $S_0\sqrt{2\Delta T} {\left|\gamma \left(u\right)\right|}^2$ is the SNR of the combined measurement at position $u$ and observation time $\Delta T$. It is possible to control the relative SNR values of the various baseline measurements used as image reconstruction data. This can be calculated by taking an a priori estimate of ${\left|\gamma \left(u\right)\right|}^2$, calling it ${\left|{\gamma }_{est}\right|}^2$, and choosing to stop observation at duration $\Delta T$ such that the signal-to-noise ratio is $1/\sigma$:

$$\Delta T={\displaystyle \frac{1}{2{\sigma }^2{S}_0^2{\left|{\gamma }_{est}\right|}^4}}, \sigma \triangleq 1/SNR$$

(9)

where $\sigma$ is the same positive constant for all $u$. Consequently, $\hat{G}\left(u,\Delta T\left(u\right)\right)$ becomes

$$\hat{G}\left(u,\Delta T\right)={\left|{\left|\gamma \left(u\right)\right|}^2+\sigma {\left|{\gamma }_{est}\right|}^2\left(N_{1u}\left(0,1\right)+i{N}_{2u}\left(0,1\right)\right)\right|}^{1/2}$$

(10)

We remark that the formulation of $\left|{\gamma }_{est}\right|$ permits the introduction of whatever a priori information on the image that might be available (overall dimensions, foreground configuration, etc.).

Before leaving this subject and for later use, we establish the formulation, based upon (10), that is closest to the traditional practice, namely the situation wherein there is no a priori information on $\gamma \left(u\right)$ for any baseline. Under this circumstance, it is obvious that the desired SNR for all baselines cannot be obtained unless

$$\left|{\gamma }_{est}\left(u\right)\right|=\underset{u}{\underbrace{\min }}\left\{\left|\gamma \left(u\right)\right|\right\}\triangleq {\left|\gamma \right|}_{\min }$$

(11)

in which case the noise model becomes

$$\begin{array}{c}\Delta T\triangleq {\displaystyle \frac{1}{2{\sigma }^2{S}_0^2{\left|\gamma \right|}_{\min }^4}}, \forall u\\ \hat{G}\left(u,\Delta T\right)={\left|{\left|\gamma \left(u\right)\right|}^2+\sigma {\left|\gamma \right|}_{\min }^2\left(N_{1u}\left(0,1\right)+i{N}_{2u}\left(0,1\right)\right)\right|}^{1/2}\end{array}$$

(12)

To close this section, we recognize the most important difference between stellar diameter determination via ICI and two-dimensional imaging by ICI. In the former, only measurements of coherence magnitude are made, while by virtue of the Van Cittert–Zernike theorem, the latter requires both magnitude and phase. Hence, phase retrieval algorithms are needed that use additional image domain information to supplement the lack of available phase information. This process must operate in the presence of very significant noise. The next section reviews the noise reducing phase retrieval (NRPR) algorithm, which has been shown [5] to be effective in handling very large amounts of noise.

3. Noise Reducing Phase Retrieval Algorithm and Reduction of Integration Times

In References [5,6], we have described a new phase retrieval algorithm and its application to imaging in the context of ICI. The algorithm recognizes that much of the noise in the averaging data is inconsistent with the image domain constraints and can be rendered harmless if both the Fourier domain and Image domain constraints can be made to intersect. The algorithm accepts the noisy coherence magnitude data and uses a relaxation technique to project these data onto a subspace wherein the image domain constraints can be satisfied. When it is run to completion for a single set of coherence magnitude data, we have shown by computational examples that the impact of much of the noise is eliminated, even for extremely large amounts of noise. For this section, it is assumed that the positions of the black pixels are known a priori.

As mentioned previously, we assume that measurements of the coherence magnitude are accumulated at points on a square grid, N pixels on the side. The image is likewise pixelated into a square grid. Thus, both the image and the coherence magnitudes compose nonnegative matrices of the same dimensions. For this discussion, it is supposed that the image consists of a bright object against a black (zero intensity) sky and that we correctly identify a boundary enclosing the bright object, such that the object cannot be translated or rotated by 180° without being clipped. The foreground and background pixels are defined as the pixel sets that are contained within and outside of, respectively, the boundary. It is assumed that the size of the background is roughly three times the size of the foreground.

It is convenient to consider both image and coherence as $N^2$-dimensional vectors. To be specific, if $g\left(k,j\right)$ denotes the two-dimensional matrix representing the image, then $g\in {\mathrm{C}}^{N^2}=vec\left(g\left(k,j\right)\right)$, where $vec\left(M\right)$ is the Kronecker algebra operator that stacks the columns of the two-dimensional matrix into a single-column vector. We further define the following notation:

$$\begin{array}{l}g\in {\mathrm{C}}^{N^2}=\mathrm{Current} \mathrm{value} \mathrm{of} \mathrm{the} \mathrm{estimated} \mathrm{image} \left(\mathrm{pixellated}\right)\\ \overline{g}\in {\mathrm{R}}^{N^2}=\mathrm{The} \mathrm{true} \mathrm{image}\\ \tau \in {\mathrm{R}}^{N^2\times N^2}=\mathrm{Projection} \mathrm{onto} \mathrm{the} \mathrm{image} \mathrm{pixels} \mathrm{constrained} \\ \hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em} \mathrm{to} \mathrm{have} \mathrm{zero} \mathrm{intensity} ({\tau }_k=1),\mathrm{while} \mathrm{all} \mathrm{other} \\ \hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em} \mathrm{pixels} \mathrm{are} \mathrm{unconstrained} ({\tau }_j=0)\end{array}$$

(13)

$$\begin{array}{l}\Im \in {\mathrm{C}}^{N^2\times N^2}= \mathrm{Discrete} \mathrm{Fourier} \mathrm{transform} (\mathrm{unitary} \mathrm{matrix}, {\Im }^{-1}={\Im }^H)\\ \overline{G}=\Im \overline{g}=\mathrm{The} \mathrm{true} \mathrm{coherence}\\ \hat{\mathrm{G}}\in {\mathrm{R}}^{N^2}=\mathrm{Measured} \mathrm{coherence} \mathrm{magnitude}\end{array}$$

(14)

In this discussion, the measured coherence magnitudes are assumed to have the form (12). Thus, the integration time is presumed to be the same for all baselines. Occasionally, we may wish to work with the two-dimensional matrix forms of $g, \overline{g}, \overline{G}$, etc. These are distinguished by an under-bar, as in $\underset{\_}{g}$, etc., and the Fourier transform of $\underset{\_}{g}$ is denoted by $\underset{\_}{\Im }\circ \underset{\_}{g}$, where $\left(\dots \right)\circ \left(\dots \right)$ indicates the Hadamard product.

The following phase retrieval algorithm [5] is considered:

$$\begin{array}{c}\mathrm{Generate} \mathrm{noisy} \mathrm{data} \mathrm{batch} \hat{G}\left(u,\Delta T\right) \mathrm{using} \mathrm{observation} \mathrm{times} \Delta T:\\ \Delta T\triangleq {\displaystyle \frac{1}{2{\sigma }^2{S}_0^2{\left|\gamma \right|}_{\min }^4}}, \forall u\\ \hat{G}\left(u,\Delta T\right)={\left|{\left|\gamma \left(u\right)\right|}^2+\sigma {\left|\gamma \right|}_{\min }^2\left(N_{1u}\left(0,1\right)+i{N}_{2u}\left(0,1\right)\right)\right|}^{1/2}\\ \mathrm{Initialize} \mathrm{g}:\\ g=\delta rand(N^2) , \left(\delta ={10}^{-6}\right) \mathrm{Each} \mathrm{element} \mathrm{of} {\Re }^{N^2}\mathrm{independently} \mathrm{and}\\ \mathrm{uniformly} \mathrm{distributed} \mathrm{in} \left[0,1\right]\\ \Downarrow \end{array}$$

(15)

(16)

where ${\tau }_{\perp }=\left[I-\tau \right]$ and $\left(\dots \right)\circ \left(\dots \right)$ denotes the Hadamard product. ${‖\dots ‖}_F$ denotes the Frobenius norm. If A is a complex-valued vector or matrix, $\left|A\right|$ denotes the matrix wherein each element of A is replaced by its absolute magnitude. Likewise, $\mathrm{Re}\left(A\right)$ is the result of replacing each element of A by its real part. If A and B are real matrices of equal dimension, then each element of $\max \left\{A,B\right\}$ is the larger of the corresponding elements of A and B. Element-by-element inequalities are denoted by double inequality symbols, e.g., A >> B, A << B, etc. Finally, $rand(N^2)$ is a vector of statistically independent, uniform random variables in $\left(0,1\right]$.

A brief explanation of the algorithm is in order. Steps B, C, D, and F are taken from the Hybrid Input/Output (HIO) algorithm of Fienup [9]. For small or zero-noise data, HIO converges rapidly, whereas other methods are prone to very slow convergence. On the other hand, if noise is more than unity, HIO fails to converge and in fact will sustain rapid oscillatory motions. Thus, steps B and E are necessary.

The fundamental input to the algorithm is a noisy batch of coherence magnitude measurements as formulated in (12). The algorithm begins with a random guess at the image such that each pixel intensity is uniformly distributed within a small interval, $\delta$. There follows the core of NRPR consisting of a recursive algorithm of six steps. Step A merely subjects the previous value of the image to Fourier transformation to produce the corresponding coherence (both amplitude and phase). Step B is crucial to noise reduction. It modifies the coherence magnitude data by discounting slightly (through the relaxation parameter $\epsilon$) the existing data values while adding $\epsilon$ times the magnitude of the latest coherence estimate. Useful values of $\epsilon$ are typically ${10}^{-5}\text{–}{10}^{-6}$. This step permits the joint convergence of both the image domain constraints and the coherence magnitude modified data that would otherwise be impossible in the presence of significant noise. An analysis of the algorithm [5] shows that from iteration to iteration, the noise in the image is inexorably reduced relative to the signal component. The remaining steps, except for E, are very similar to the HIO algorithm of Fienup [9]. Step C takes $G^{\prime }=\Im \mathrm{g}$ and preserves its phase but replaces its magnitude with the modified coherence magnitude data to obtain the preliminary coherence estimate $G_p$. This is essential for the gradual reduction of the constrained pixel intensities. Step D calculates the corresponding preliminary image, $g_p$. Step E speeds convergence by ensuring that the constrained pixel intensities are real-valued, and the unconstrained intensities are real and nonnegative. Finally, step F reduces the intensities of the constrained pixels in accordance with the HIO algorithm. The parameter $\beta$ equals 0.7. After each iterate of the recursive algorithm, the 1-norm of the constrained pixel intensities, $E_g={‖\tau g‖}_1$, is computed. In this work, convergence to an image having errors indetectable by visual inspection are obtained with $E_g<{10}^{-3}$.

In Reference [5], various properties of the algorithm were explored. In particular, when run to convergence, the algorithm is equivalent to an operator that projects the coherence magnitude measurement data into the subspace that is consistent with the image domain constraints, thereby eliminating a large portion of the coherence magnitude measurement noise. These conclusions do not depend upon the noise model.

Before leaving this section, we determine the value of the reciprocal SNR, $\sigma$. Ref. [8] gives an overview of the imaging examples using the integration times as calculated by Brown and Twiss. These results reveal that the small features of the image converge first. Using the Hirschman entropic uncertainty analysis [10,11,12] shows that the Brown–Twiss integration time is much larger than needed. To correct this, $\sigma$ must include terms involving the constrained pixels. Also, to treat all possible factors, the sum of the number of constrained pixels must appear in all of the terms $\left(N_{1u}\left(0,1\right)+i{N}_{2u}\left(0,1\right)\right)$ in $\hat{G}\left(u,\Delta T\right)$. Finally, since $\Delta T$ is a constant for all coherence magnitude measurements, we conclude the following:

$$\begin{array}{c}\sigma =N_{C}C\\ N_C\triangleq \mathrm{The} \mathrm{number} \mathrm{of} \mathrm{constrained} \mathrm{pixels}\\ C\triangleq \mathrm{A} \mathrm{constant} \sim \mathrm{O}\left(1\right)\end{array}$$

(17)

Now, we proceed to determine the value of $C$. The validity of (17) is explored using randomly generated image examples capable of encompassing a multitude of image sizes. As mentioned previously, we assume that there is an illuminated foreground that contains the object to be imaged and that it is centered within the field of view, while the background contains the constrained black pixels (${\tau }_k=1$). The boundary of the foreground/background can be almost any convex contour, but as it turns out, the most convenient shape is a square with $B_x$ pixels on the side. This is the contour that separates the unconstrained pixels in the foreground from the black pixels in the background. Reference [7] designates this contour as the box.

Now, we consider the constant C. With the above foreground/background prescription, we run the NRPR algorithm (15–16) for many box sizes, some of which are depicted in Figure 1. To prevent 180-degree rotation of the image during NRPR convergence, a dark square “tab” is placed in the upper left corner of the foreground, within the box.

The intensity of each pixel of the foreground is uniformly distributed in $\left(0,1\right]$, with all pixels statistically independent. Model (12) is implemented through the precise calculation of ${\left|\gamma \right|}_{\min }=\underset{u}{\underbrace{\min }}\left[\left|\gamma \left(u\right)\right|\right]$ in every case. For each $B_x$, the algorithm is run 20 times for a selected $\sigma$ value, and the convergence is checked to see if the correct convergence is achieved in at least 18 out of 20 cases. We note that NRPR always converges so that the background constraints are satisfied, while “correct” convergence is deemed to occur if the foreground image is in agreement with the actual image to the extent that the 1-norm of the error is less than 0.001. Failure to correctly converge is accompanied not only by significant foreground error but also a very noticeable decline in the average intensity of the foreground.

Using this definition of convergence failure, we start with $\sigma =1$, then increase $\sigma$ until correct convergence in more than 10% of cases fails (in all cases when $\sigma =1$, there was 100% convergence). Then, the next lowest value of $\sigma$ is recorded. This process is undertaken for each value of $B_x$. We then summarized the results in Figure 2, which shows the largest values of $\sqrt{\sigma }$ for which convergence succeeds at least 90% of the time as a function of $B_x$. The resulting variation of $\sigma$ with $B_x$ is approximately as follows:

$$\sigma \cong {\left(1.2\pm 0.011\right)}^2{B}_x^2$$

(18)

Of course, by the reasoning of (17), the increase in sigma is due to the number of image domain constraints (the number of constrained pixels), denoted here by $N_C$. By virtue of the foreground/background proportions, $N_C=8B_x^2$. Consequently, we have the following:

$$\begin{array}{c}\sigma \triangleq {\displaystyle \frac{1}{SN{R}_C}}=\frac{1}{8}{\left(1.2\pm 0.011\right)}^2{N}_C\cong {\displaystyle \frac{N_C}{4\sqrt{2}}}\\ \Delta T\cong {\displaystyle \frac{1}{S_0^2{\left|\gamma \right|}_{\min }^4}}{\left({\displaystyle \frac{4}{N_C}}\right)}^2\end{array}$$

(19)

where $SN{R}_C$ denotes the minimum SNR required for NRPR convergence. This result substantially validates [6]. Note that in comparison with the conventional value of $\Delta T$ corresponding to a desired SNR of unity, the integration time is diminished by the factor $N_C^2/32$. For a relatively small image that is only 100 pixels on a side, this factor amounts to slightly more than two hundred thousand. Thus the $\sigma$ value is an asymptotic result.

It is also evident that the above relations lead to a simple criterion for NRPR convergence, namely that the product of the $SN{R}_C\triangleq 1/\sigma$ and the number of constraints has a nonzero, positive lower bound:

$$N_{C}SN{R}_C\ge 4\sqrt{2}$$

(20)

To test whether or not the above integration time results are sensitive to image content, we examined several non-random images using the same foreground/background arrangement and computational procedure as is described following Equation (19). Figure 3 depicts the noise-free images being tested, where only the foreground in each case is shown. Corresponding to each recovered image, the stars indicate the largest values of $1/\sqrt{\sigma }$ for which convergence succeeds at least 90% of the time as a function of $B_x$. The straight line shows the previously derived estimate for $1/\sqrt{\sigma }$. At least for the cases tested, there appears to be little sensitivity to image content.

The conclusion to this section is that the formula for the computation of the coherence magnitude values is as follows:

$$\begin{array}{c}\Delta T\triangleq {\displaystyle \frac{1}{S_0^2{\left|\gamma \right|}_{\min }^4}}{\left({\displaystyle \frac{4}{N_C}}\right)}^2, \forall u\\ \hat{G}\left(u,\Delta T\right)={\left|{\left|\gamma \left(u\right)\right|}^2+\left({\displaystyle \frac{N_C}{4\sqrt{2}}}\right){\left|\gamma \right|}_{\min }^2\left(N_{1u}\left(0,1\right)+i{N}_{2u}\left(0,1\right)\right)\right|}^{1/2}\end{array}$$

(21)

Note that the ratio of the above integration time to the Brown–Twiss calculation is as follows:

$${\displaystyle \frac{{\left(\Delta T\right)}_{NRPR}}{{\left(\Delta T\right)}_{Brown-Twiss}}}\cong {\left({\displaystyle \frac{4}{N_{C}SN{R}_d}}\right)}^2$$

(22)

where $SN{R}_d$ is typically 10 to 100. Hence, this ratio of the two is many orders of magnitude smaller.

A limitation of the present section is that the set of constrained pixels is determined a priori. To apply ICI to image space objects of unknown configuration, we must be able to discover the constrained set without prior knowledge (except for the fact that we are dealing with a bright object against a black sky). This is treated in the following section.

4. NRPR Characteristics and Stochastic Search

Having obtained the NRPR algorithm with a box size along with its appropriate integration time, we can run NRPR to completion for multiple box sizes. The outcome of each iterate is a hypothesis concerning the correct, zero-noise image. In this section, we show how to embed NRPR within a stochastic search mechanism that quickly discovers the true image. We begin with the stochastic search algorithm approach introduced in [7] and shown in Figure 4.

The first step is to collect the intensity fluctuation data and compute all the cross-correlations, using the correct integration time. The stochastic search algorithm uses only these data. The first stage of the search algorithm (shown in green) sets up a “Box” of size $B_x$. The second loop (shown in red) runs NRPR up to n > 1 times. Each run uses an initial guess of the image such that the intensities of all pixels are statistically independent and uniformly distributed in $\left[0,\delta \right)$, where $\delta$ is a small real and nonnegative number. The second part of the loop ends by updating the image correlation matrix. Then (shown in black), we test the correlation matrix to see if there is a pair of fully correlated images among the n NRPR trials (allowing for 180-degree rotations and translations in the field of view). If this is not the case, then we set up a new box size and try again. If there are two correlated images, then we stop the stochastic search because as we will show, both correlated images are the zero-noise image. Generally, excellent image quality is attained when the image domain constraint error, $E_{\tau }$, is less than ${10}^{-3} {‖g‖}_{\infty }$.

Both the analysis [6,7] and thousands of computational runs show that the NRPR algorithm always converges to some $g$. Thus, we may consider NRPR as a mapping from the intial values $\tau , g_0, \mathrm{and} {\hat{G}}_0$ to the converged result, denoted by $g_c$:

$$g_c={\mathcal{N}}_g\left(\tau ,{\hat{G}}_0,g_0\right)$$

(23)

as well as a mapping from $\tau , g_0, \mathrm{and} {\hat{G}}_0$ to $G_c$:

$$\begin{array}{c}{G}_c={\mathcal{N}}_G\left(\tau ,{\hat{G}}_0,g_0\right) \\ {\mathcal{N}}_G\left(\tau ,{\hat{G}}_0,g_0\right)=\Im \left[{\mathcal{N}}_g\left(\tau ,{\hat{G}}_0,g_0\right)\right]\end{array}$$

(24)

One expects that the randomness of $g_0$ will result in the randomness of $g_c$ so that $g_c$ may not be the true, noise-free image $\overline{g}$. However, there is one instance where $g_c=\overline{g}$ independently of $g_0$, and that is when $\tau$ equals the true $\tau$; we call it $\overline{\tau }$. This is illustrated in Figure 5. For all cases the coherence magnitude data are fixed, and the results are all obtained after 500 iterates. The upper left shows $\tau$ equal to the true constraint matrix. The top sequence of results shows a sample of independent trials of $g_c=\mathcal{N}\left(\tau ,{\hat{G}}_0,g_0\right)$ (taken from 10,000 samples), obviously duplicating the true image repeatedly. In contrast, the lower left shows an erroneous specification of $\tau$. Repeated trials of $g_c=\mathcal{N}\left(\tau ,{\hat{G}}_0,g_0\right)$, using different random samples of $g_0$ each time result in randomly varying images. This basic characteristic of NRPR suggests that at the very least, the correct background region can be discovered by trial and error using NRPR as a test. Of course, a purely trial-and-error approach would entail the evaluation of an enormous number of possible trials (perhaps as much as $2^{N^2}$), especially for any reasonably large image. A quicker technique rests upon another characteristic feature of NRPR convergence, namely the following: Given random trial values of $\tau$ (that could include $\tau =0$), there is a nonzero probability that the repeated trials will produce two or more identical repetitions of the correct image. Once such a repetition occurs the identical images produced can be identified as the correct image. The probability of such image repetition grows with the number of correctly assigned elements of $\tau$.

Therefore, there is the possibility of identifying the correct image domain constraints with relatively few random trials.

We illustrate the algorithm’s characteristics using the simple spacecraft image first introduced in [5] and introduced here as Figure 6. This image is set in a total field of view of 90 by 90 pixels. The NRPR parameters are specified to be as follows:

$$\begin{array}{c}\epsilon =2\times {10}^{-5}, \beta =0.7\\ \mathrm{Number} \mathrm{of} \mathrm{iterations}=1000\end{array}$$

(25)

The choice of these algorithm parameters warrants some explanation. First, consider epsilon. If the epsilon is less than or equal to zero, NRPR devolves into HIO and generally does not converge for large values of noise. Thus, epsilon must be positive. An enormous number of trials of NRPR on a great many different images has shown excellent performance for epsilon in the range epsilon = $2.0\times {10}^{-5}$ to $1.0\times {10}^{-6}$.

The first step in developing this example is to calculate the raw coherence magnitude data batch that will be used repeatedly in every sample run. This is given by ${\hat{G}}^2\left(u_k,\Delta T\right)$. The value for ${\hat{G}}^2\left(u_k,\Delta T\right)$ is the initial value $\hat{G}$, which is subsequently adjusted via Step B of NRPR. For this example, we estimate that ${\left|\gamma \right|}_{\min }=1.0\times {10}^{-3}$ (it turns out that the minimum value is ${\left|\gamma \right|}_{\min }=0.79\times {10}^{-3}$). The number of constraints, $N_C$, is given as a function of the box size, denoted by $B_X$ and by ${\left(90\right)}^2-B_X^2$.

With the raw coherence magnitude data in hand and the box size specified, the procedure is to compute a sequence of NRPR cases (each run for 1000 iterations). Each member of this sequence is an instance of $g_c={\mathcal{N}}_g\left(\tau ,{\hat{G}}_0,g_0\right)$. This sequence terminates when there are two images that are identical (aside from the trivial ambiguities and within 0.001 relative error), whereupon these are deemed the true, noise-free image.

We begin by showing a typical example of the image sequence for $B_X=44$, which is a reasonable starting value. The results shown in Figure 7 are quite surprising. There are two distinct categories of image: a set of random and non-repeatable images (images 1, 2, 4, and 5) and two identical images (numbers 3 and 6) that perfectly match (within the above mentioned error criteria) the correct noise-free image. In the latter, not only are the constrained pixels outside the box correctly prescribed, but all the truly black pixels within the box are also rendered correctly, as are the unconstrained pixels of the noise-free image.

The above phenomena can be explained by the structure of NRPR considered as a discrete time dynamical system with state $g\left(k\right)\in {ℝ}^{N\times N}, k=1,2,\dots ,\infty$, where k is the iteration number. Here, we distinguish between the assumed value of $\tau$ and the value of $\tau$ that includes all of the black pixels belonging to the true, noise-free image, denoted by ${\tau }_{*}$. There are two primary noise reduction processes operating in NRPR. The first is step F, which reduces the noise in the constrained region of the state space. Assuming that the specified box is not smaller than the bright object, then $‖\tau g‖\underset{k\to \infty }{\underbrace{\to }}0$. The second mechanism for noise reduction is Step B. The application of Step B distinguishes two subsets of the initial values of the pixels denoted by $g_0$. One subset evolves via the action of Step B into the noise reduction of pixels $\left(k,j\right)$ that do not contain $\left(k,j\right):{\left[{\tau }_{\perp }\circ {\tau }_{*\perp }\right]}_{k,j}=1$. This set converges to an image that retains noise in the foreground of the image, as illustrated by the random configurations in images 1, 2, 4, and 5 of Figure 7. The second set, however, is the one under which Step B reduces the noise for the entirety of the unconstrained set $\left(k,j\right):{\left[{\tau }_{\perp }\right]}_{k,j}=1$. In this case, the whole image converges to the noise-free image (as in images 3 and 6), and the final result is independent of the initial noise in the coherence magnitude measurements. A few more deductions can be made on the basis of this discussion. For example, the probability of obtaining two identical images from a sequence of runs should increase as the box size decreases as long as the bright object can be contained in the box. On the other hand, if the box is too small to contain the bright object, we have an irreconcilable conflict between the image-domain and Fourier-domain constraints so that there can only be random images (as we illustrate below).

Clearly, a trained human operator can use the above stochastic search method to rapidly determine the correct constraints and the image. For example, the results for $B_X=70$ already indicate the approximate size of the object, so a skilled technician would go immediately to $B_X=30$ without passing through the intermediate steps and then secure the correct image.

To illustrate this example further, we proceed to examine the statistical characterisitics of the number of trials (for a given box size) needed to secure the noise-free image. The main element determining the computational labor is the number of NRPR trials needed to identify the noise-free image for a given box size. Here, we show typical statistical estimates using histograms calculated in [7]. For each box size, we obtain 500 statistical samples. Each sample consisting of repeated NRPR runs, with statistically independent initial conditions, provides the number of independent NRPR trials, $R$, needed to obtain two images that have a normalized correlation coefficient within ${10}^{-3}$ of unity.

Figure 8 shows the resulting histograms of the computed number of NRPR trials, $R$, for box sizes of 70, 60, 50, 40, and 30. These histograms, denoted by $\mathrm{Pr}\left(R\right)$, display for each R the probability of obtaining the noise-free image for each repetition, R, of the stochastic search, noting that each repetition ends with the second noise-free image. In practice, a reasonable starting value of box size would be roughly half the size of the field-of-view, corresponding in this case to $B_X=50$. As we reduce the box size down to the autocorrelation of the true foreground, the lowest histogram in the figure shows that all samples arrive at the noise-free image in just two iterations of NRPR. When we consider that a purely stochastic search would take on the order of $2^{N^2}$ NRPR runs, it is clear that the stochastic search algorithm is highly efficient and that the data processing time for the combined stochastic search/NRPR algorithm is quite modest.

Another observation is that for $B_X$ below 30 and greater than 16, the correct convergence to the noise-free image again requires only two iterations of NRPR. It is to be noted that as long as the box is large enough to contain the foreground, the imaged object will be located within the box. However, for 16 and below, the box is too small to contain the bright object, and therefore, NRPR cannot correctly converge to a pattern that is entirely distributed within the box. In fact, repeated trials produce completely different image patterns that remind one of a disordered fragmentation that is widely distributed beyond the confines of the box. Figure 9 illustrates this behavior. The main conclusion is that the algorithm informs the user when the specified box size is too small.

5. Analysis and Refinement

Having explored the features of the stochastic search method via computational experimentation in Section 4, we review the supporting theorems discussed in Reference [8] to validate the previous results. This will also result in an even faster method of image identification. To illustrate and validate the analytical derivations presented in [8], we adopt a more complex test image, shown in Figure 10. This is a rendering of a GOES 16-type geophysical satellite at geostationary orbit with 20 cm resolution. For the present investigation, the values of the parameters in the above are fixed at

$$\beta =0.7, \epsilon =2\times {10}^{-5}, \delta ={10}^{-6}$$

(26)

and it is customary practice to choose the field of view size, $N$, so that it is more than three times the size of the illuminated object.

The first step is to recast the Equations (15) and (16) into a discrete-time dynamic system such that all the principal quantities are indicated by the sequence of integers $k=0,1,\dots \infty$. Consider the equations pertaining to $k=0 \mathrm{and} 1$:

$$\begin{array}{l}{\hat{G}}_{kj}(0)\triangleq {\left|{\left|{\gamma }_{kj}\right|}^2+\frac{N_C}{4\sqrt{2}}{\left|{\gamma }_{est}\right|}^2{N}_{kj}\left(0,1\right)\right|}^{1/2}\\ \mathrm{g}\left(0\right)=\delta rand\left(N,N\right), \delta \cong {10}^{-6}\\ \hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\Downarrow \\ G\left(1\right)=\Im \left[\delta rand\left(N,N\right)\right]\\ \hat{G}\left(1\right)=\left(1-\epsilon \right)\hat{G}\left(0\right)+\epsilon \delta \left|e^{i\pi \left[2rand\left(N,N\right)-1\right]}\right|=\hat{G}\left(0\right)+O\left(-\epsilon +\epsilon \delta \right)\\ \tau \circ g\left(1\right)=\tau \circ \left\{g\left(0\right)-\beta \mathrm{Re}\left[{\Im }^H\left(\hat{G}\left(0\right)\circ e^{i\pi \left[2rand\left(N,N\right)-1\right]}+O\left(-\epsilon +\epsilon \delta \right)\right)\right]\right\}\\ {\tau }_{\perp }\circ g\left(1\right)={\tau }_{\perp }\circ {\mathrm{Re}}_{+}\left[{\Im }^H\left(\hat{G}\left(0\right)\circ e^{i\pi \left[2rand\left(N,N\right)-1\right]}+O\left(-\epsilon +\epsilon \delta \right)\right)\right]\end{array}$$

(27)

where the function $rand\left(N\times N\right)$ is a Kronecker vector having statistically independent elements that are uniformly distributed in $\left[0,1\right)$, as defined previously. Because $\epsilon$ and $\delta$ are very small, the residual terms are discarded for all k. This leads to the discrete dynamic system:

$$\begin{array}{l}{\hat{G}}_k\left(1\right)\cong {\hat{G}}_k\left(0\right)\triangleq {\left|{\left|\gamma \right|}_{\min }^2+\frac{N_C}{4\sqrt{2}}{\left|\gamma \right|}_{\min }^2{N}_k\left(0,1\right)\right|}^{1/2}\\ \tau \circ g\left(1\right)=\beta \tau \circ \mathrm{Re}\left[{\Im }^H\left(\hat{G}\left(0\right)\circ e^{i\pi \left[2rand\left(N,N\right)-1\right]}\right)\right]\\ {\tau }_{\perp }\circ g\left(1\right)={\tau }_{\perp }\circ {\mathrm{Re}}_{+}\left[{\Im }^H\left(\hat{G}\left(0\right)\circ e^{i\pi \left[2rand\left(N,N\right)-1\right]}\right)\right]\\ \hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\Downarrow \mathrm{k}=1,\dots ,\infty \\ \hat{G}\left(k+1\right)=\left(1-\epsilon \right)\hat{G}\left(k\right)+\epsilon \left|\Im \left[\mathrm{g}\left(k\right)\right]\right|\\ \tau \circ g\left(k+1\right)=\tau \circ \left\{g\left(k\right)-\beta \mathrm{Re}\left[{\Im }^H\left(\hat{G}\left(k+1\right)\circ e^{i\arg \left(\Im \left[\mathrm{g}\left(k\right)\right]\right)}\right)\right]\right\}\\ {\tau }_{\perp }\circ g\left(k+1\right)={\tau }_{\perp }\circ {\mathrm{Re}}_{+}\left[{\Im }^H\left(\hat{G}\left(k+1\right)\circ e^{i\arg \left(\Im \left[\mathrm{g}\left(k\right)\right]\right)}\right)\right]\end{array}$$

(28)

Now, retaining the Kronecker vector notation, $g\left(k\right)$ is represented by the column vector:

$$g\left(k\right)=\left(\begin{array}{c}{g}_b\left(k\right)\\ g_f\left(k\right)\end{array}\right),\hspace{1em}{g}_b\left(k\right)\in {ℝ}^{N^2-B_x^2},\hspace{1em}{g}_b\left(k\right)\in {ℝ}^{B_x^2}$$

(29)

In other words, $g_b\left(k\right)$ is a vector composed of the matrix elements of the background pixels, and $g_f\left(k\right)$ is the vector composed of the matrix elements of the foreground pixels. Corresponding to this is the following:

$$G\left(k\right)\triangleq \Im \left[\mathrm{g}\left(k\right)\right]=\left[\begin{array}{cc}{F}_b& F_{bf}\\ F_{bf}^H& F_f\end{array}\right]\left(\begin{array}{c}{g}_b\left(k\right)\\ g_f\left(k\right)\end{array}\right),\hspace{1em}\hat{G}\left(k\right)=\left(\begin{array}{c}{\hat{G}}_b\left(k\right)\\ {\hat{G}}_f\left(k\right)\end{array}\right)$$

(30)

$\left[\begin{array}{cc}{F}_b& F_{bf}\\ F_{bf}^H& F_f\end{array}\right]$ is the unitary matrix corresponding to the Fourier transform operator in the Kronecker notation. Finally, $\tau$ is represented by

$$\tau =\left[\begin{array}{cc}{I}_{N^2-B_x^2}& 0\\ 0& 0\end{array}\right]$$

(31)

Reference [8] commenced the computation of an asymptotic approximation of the background, introducing the averages of $g_b\left(k\right)\in {ℝ}^{N^2-B_x^2}, g_b\left(k\right)\in {ℝ}^{B_x^2}$ as defined below:

$$〈g_b\left(k\right)〉\triangleq \frac{1}{N^2-B_x^2}{\displaystyle \sum _{m=1}^{N^2-B_x^2}{\left|g_b\left(k\right)\right|}_m} 〈g_f\left(k\right)〉\triangleq \frac{1}{B_x^2}{\displaystyle \sum _{m=1}^{B_x^2}{\left|g_f\left(k\right)\right|}_m}$$

(32)

To the first order, the maximum estimate for the limit on $〈g_b\left(k\right)〉$ can be found to be

$$G_f\left(k\right)\cong \left(1+\alpha \epsilon k\right)\left|{\hat{G}}_f\left(k\right)\right|,\hspace{1em}\alpha \cong 10$$

(33)

The resulting asymptotic approximation is

$$\begin{array}{c}〈g_b\left(k+1\right)〉=\left(1-\frac{1}{4}\beta \right)〈g_b\left(k\right)〉+\frac{1}{2}\beta \frac{A}{N^2\left(1+\alpha \epsilon k\right)}〈g_b\left(0\right)〉\\ 〈g_b\left(k=0\right)〉\cong 0.05, A={\left(30\right)}^2, N=140, \alpha =10\end{array}$$

(34)

The computations for the first 40 iterations are shown in Figure 11 and Figure 12, which give the complete 7000 iterations.

A comparison of the predicted and computed results in Figure 11 and Figure 12 establishes that the averaged background intensity is reduced by an order of magnitude at the outset and then levels off beyond iterate $k=k_T\cong 25$ to form a floor that does not vary appreciably beyond $k=k_T$ and within several hundred iterations thereafter. These features match the computed results rather well, especially for the immediate drop-off for $k<k_T$, where it is established that $〈g_b\left(k\right)〉$ is very weakly dependent on $B_x$. Figure 12 shows the complete averaged background from k = 1 to k = 7000. The conclusion of this derivation is that after $k=k_T$, $g_b\left(k\right)$ is so suppressed relative to $g_f\left(k\right)$ that its influence on the foreground may be neglected. Thus, we set $g_b\left(k\right)\cong 0, \mathrm{and} \left[\begin{array}{l}{G}_b\left(k\right)\\ G_f\left(k\right)\end{array}\right]\cong \left[\begin{array}{cc}0& 0\\ 0& F_f\end{array}\right]\left(\begin{array}{c}{g}_b\left(k\right)\\ g_f\left(k\right)\end{array}\right)$. In addition, for the first few iterates, the initial value of $e^{i\arg \left(F_f{g}_f\left(k\right)\right)}$ is hardly affected so that we are left with

$$\begin{array}{l}\arg \left(F_f{g}_f\left(k_0\right)\right)\cong \pi \left[2rand\left(B_x,B_x\right)-1\right]\\ {\hat{G}}_f\left(k+1\right)={\hat{G}}_f\left(k\right)+\epsilon \left(\left|F_f{g}_f\left(k\right)\right|-{\hat{G}}_f\left(k\right)\right)\\ g_f\left(k+1\right)={\mathrm{Re}}_{+}\left\{F_f^H\left[{\hat{G}}_f\left(k+1\right)\circ e^{i\arg \left(F_f{g}_f\left(k\right)\right)}\right]\right\}\end{array}$$

(35)

At this point, References [8,13,14] executed a complex derivation to transform the above system into two quantities.

${\Psi }_{m n}\left(k\right)\triangleq {\left\{{\Im }_f^H\left[{\hat{G}}_f\left(k\right)\right]\right\}}_{m n} ,{\hat{\Lambda }}_{m n}\left(k\right)\triangleq {\left\{{\Im }_f^H\left[e^{i\arg \left(F_f{g}_f\left(k\right)\right)}\right]\right\}}_{m n}$ defined on the finite support set: $\left\{-B_x,-B_x+1,\dots ,B_x-1,B_x\right\}\times \left\{-B_x,-B_x+1,\dots ,B_x-1,B_x\right\}$. The transformed equations are

$$\begin{array}{c}\hat{\Lambda }\left(k_0\right)=rand\left(B_x,B_x\right)\\ \Psi \left(k+1\right)-\Psi \left(k\right)=\epsilon \left(g_f\left(k\right)-\Psi \left(k\right)\right)\\ \hat{\Lambda }\left(k\right)={\Im }_f^H\left[e^{i\arg \left(F_f{g}_f\left(k\right)\right)}\right]\\ g_{f m n}\left(k+1\right)={\mathrm{Re}}_{+}{\left\{\Psi \left(k\right)\ast \hat{\Lambda }\left(k\right)\right\}}_{m n}\end{array}$$

(36)

where $\left(\dots \ast \dots \right)$ is the correlation operator. Next, we have several proofs of the nonnegativity of $\Psi \left(k\right), \Lambda \left(k_0\right) {\mathrm{and} \mathrm{g}}_f$:

Theorem 1.

$\Psi \left(k\right)\triangleq {\Im }_f^H\left[{\hat{G}}_f\left(k\right)\right]$ is the real, nonnegative autocorrelation of ${\Im }_f^H\left[\sqrt{{\hat{G}}_f\left(k\right)}\right]$.

Proof. 

Since ${\hat{G}}_f\left(k\right)$ is nonnegative, we have ${\hat{G}}_f\left(k\right)=\sqrt{{\hat{G}}_f\left(k\right)}{\left(\sqrt{{\hat{G}}_f\left(k\right)}\right)}^H$; thus, the result follows Parseval’s theorem. □

Corollary 1.

Since both $g_f\left(k\right) \mathrm{and} \Psi \left(k\right)$ are real and nonnegative, so is $\hat{\Lambda }\left(k\right)$.

Proof. 

$g_f\left(k+1\right)=\left(\Psi \left(k\right)\ast \Lambda \left(k\right)\right){=\mathrm{Re}}_{+}\left(\Psi \left(k\right)\ast \Lambda \left(k\right)\right)=\left(\Psi \left(k\right)\ast {\mathrm{Re}}_{+}\left(\Lambda \left(k\right)\right)\right)$. □

Thus, $\Psi \left(k\right)$, $\hat{\Lambda }\left(k\right)$, and g are nonnegative and real. The following shows a necessary condition for convergence to the noise-free image.

Theorem 2.

If the box contains the entirety of the region of support of $\Psi \left(k\right)$, then (a) the correlation, and $\left(\Psi \left(k\right)\ast \hat{\Lambda }\left(k\right)\right)$ is maximum; (b) this is a necessary condition for g to converge to the noise-free image.

Proof. 

Earlier in the analysis (see Equation (33)), it follows that $\Psi \left(k\right)$ converges to a maximum value. Given this, $\Psi \left(k\right)$ is bijective to ${\hat{G}}_f\left(k\right)$ and $e^{i\arg \left({\Im }_f{g}_f\left(k\right)\right)}$. Then, the main conclusion follows from the work of Klibanov [15,16]. □

Theorem 3.

If the box contains the entirety of the region of support of the noise-free image, $\overline{g}$, and no less, then this is a necessary and sufficient condition for the convergence to the noise-free image.

Proof. 

The region of support of $\Psi \left(k\right)$ encompasses that of $\overline{g}$, implying the loss of constraints on the elements of $\hat{\Lambda }\left(k\right)$. Since a maximum cannot be reduced via the elimination of constraints, the stated condition remains necessary. Then, in view of the analysis in [8], the stated condition is necessary and sufficient. □

Comment. 

A multitude of computations indicate that if the regions of support of $\Psi \mathrm{and} \hat{\Lambda }$ overlap to the extent of 50% of area, this is almost invariable that g converges to the noise-free image. Thus, with the probability of nearly one, we may relax Theorems 2 and 3.

The following will be of use:

Theorem 4.

If the box is too small to completely contain the noise-free image, then NRPR cannot converge to the noise-free image. Furthermore, the converged NRPR run exhibits illuminated pixels outside of the box.

Proof. 

The fundamental operation of NRPR is that it simultaneously attempts to satisfy the constraints imposed by the coherence magnitude data and the image domain background constraints. These are in direct conflict so that correct convergence is impossible. □

Now, in Figure 13, we illustrate the fact that $\Psi \left(k\right)$ and $\hat{\Lambda }\left(k\right)$ occupy statistically independent positions within the foreground box, and the brighter portions of $\Psi \left(k\right)$ and $\hat{\Lambda }\left(k\right)$ encompass the region of support of the true image (or the region of support for the possible convolutions of the true image). One notices from Figure 13 that there is almost no overlap between $\Psi \left(k\right)$ and $\hat{\Lambda }\left(k\right)$. In conformity with Theorems 2 and 3 and the comment above, Figure 13 shows the convergence to a poor image, where it is apparent that the magnetometer boom is on the wrong side of the satellite. These results show that with high correlation and good image quality, there is significant overlap between $\Psi \left(k\right)$ and $\hat{\Lambda }\left(k\right)$.

The foregoing observations suggest a rather simple proof of the stochastic algorithm for high correlation and convergence to the noise-free image. This is summarized in Figure 14 as follows:

We can now construct a complete model for the probability of the zero-noise image convergence as a function of the repetitions, R, of the stochastic search algorithm for a given box size. The first step is a model for the probability for a correct convergence on any single trial as a function of the box size. This function, denoted here by $\mu$, involves both $\sqrt{A}/N$ and $B_x/N$. For the situation on the right side of Figure 14, since the relative positions within the box are completely random, the probability of an overlap between $\Psi \left(k\right)$ and $\hat{\Lambda }\left(k\right)$ can be approximated simply as the ratio of the areas $A$ and $B_x^2$, i.e., $A/B_x^2$. Note that this satisfies Theorem 3 when the situation on the right side of Figure 14 holds. Finally, when $B_x^2\le A$, Theorem 4 is satisfied. In summary, we derive the following:

$$\mu \left(B_x\right)\cong \left\{\begin{array}{l}0 , B_x<\sqrt{A} \\ {\displaystyle \frac{A}{B_x^2}} , B_x\ge \sqrt{A} \end{array}\right.$$

(37)

Figure 15 shows the above formula rendered as the blue line. The red stars represent the percentage of correct images out of one hundred trials as calculated by NRPR. The agreement is close. Also, there are a few correct images even for the case wherein the box size coincides with the whole field of view (in other words, the number of constraints are equal to zero).

It must be emphasized that this expression is the probability of NRPR attaining the correct image in a single trial, but a successful determination of the true image via the stochastic search method requires the occurrence of two identical and correct images within R independent trials, where the last trial produces the second correct image in the sequence of R trials. Let us replace $\mathrm{Pr}\left(R\right)$ in Section 4 with $p\left(n,\mu \right)$, the probability that the correct image is discovered in n trials given that $\mu$ is the probability of obtaining the correct image in one trial. The following theorem displays the required result:

Theorem 5.

The probability of obtaining the correct image via the occurrence of two correlated images in n independent trials is

$$p\left(n,\mu \right)=\left(n-1\right){\mu }^2{\left(1-\mu \right)}^{n-2}$$

(38)

Proof. 

In the stochastic search algorithm, the $n^{th}$ trial is defined as the second successful correct image, thus

$$p\left(n,\mu \right)=\mu \times \left\{\mathrm{Probability} \mathrm{that} \mathrm{there} \mathrm{is} \mathrm{one} \mathrm{successful} \mathrm{trial} \mathrm{in} \mathrm{the} \mathrm{first} n-1 \mathrm{trials}\right\}$$

The above probability has the form of a Bernoulli distribution since each trial is independent. Therefore, we derive the following:

$$p\left(n,\mu \right)=\mu \times {\displaystyle \frac{\left(n-1\right)!}{1!\left(n-2\right)!}}\mu {\left(1-\mu \right)}^{n-2}=\left(n-1\right){\mu }^2{\left(1-\mu \right)}^{n-2}$$

(39)

□

Figure 16 shows the probability distributions pertaining to the test image for a range of box sizes. These results verify that the probabilities of identifying the correct image via the stochastic search process are quite high. Figure 16 is also in line with the computational results shown in [8]. Furthermore, the progression of distributions from large to small box sizes suggests the following process. Having made a rough estimate of the object size, one starts with a box size that is twice the estimate. As seen in the third from the top distribution in Figure 16, 90% of the probability mass is contained in the first 20 trials, and the maximum probability is only 5 trials. Thus, one would set the maximum set of trials at approximately 10. In the event of no success, one should reduce the box size by approximately a quarter until two identical images occur. If one reduces the box size to less than the object size, then by Theorem 4, NRPR will not converge properly. According to [8], such an event can be recognized by the fact that the distribution of intensity in the image plane is several times the box size and the object size, in which case one must enlarge the box size slightly and continue to completion. In the next section, a refinement to the above process is introduced that exploits the reverse procedure of starting with a too small box size and proceeds to enlarge the box size.

6. A Refinement of the Stochastic Search Algorithm

Theorems 3, 4, and 5 motivate the reversal of the search process. In this refinement, one starts with a box size significantly smaller than the smallest box that can contain the illuminated object and then progressively enlarges the box until success is obtained. To understand the implications of this procedure, we illustrate with the test image.

Figure 17 shows the sequence of images produced by NRPR starting with $B_x=20$ and proceeding to $B_x=20, 22, 24, 26 \mathrm{and} 28$. These first five images show what happens when the box size is smaller than the object. The large halo of intensity surrounding the object is due to the fact that in each cycle of calculation NRPR attempts to impose Fourier domain constraints and then undoes those constraints by attempting to impose image domain constraints. The lower row of images illustrates the behavior when the box just contains the true image. The process for the identification of the noise-free image should end with the two images on the lower left of the figure, but we show additional images, all with $B_x=30$, to emphasize the stark contrast in behavior once the threshold of the smallest box allowed by Theorem 3 is crossed. This behavior reveals a significant advance for ICI because by starting with too small boxes and increasing the box size rather than vice-versa as in [8], very few box sizes must be tested, and once the box size is just that of the object, it takes only two trials to secure the true image.

This modified stochastic search algorithm also has a beneficial effect on the necessary integration time required. Note from Equation (27) that the initially measured coherence magnitudes, $\hat{G}\left(u_k,\Delta T\right)={\left|{\left|\gamma \left(u_k\right)\right|}^2+\frac{N_C}{4\sqrt{2}}{\left|{\gamma }_{est}\right|}^2{N}_{u_k}\left(0,1\right)\right|}^{1/2}$, have noise terms proportional to the number of image domain constraints, $N_C$. This quantity is inversely proportional to the box size, and in Equation (35) of [11], the ratio of the integration time needed by NRPR to the integration time as conventionally estimated by Brown and Twiss is ${\left(\Delta T\right)}_{NRPR}/{\left(\Delta T\right)}_{conventional}\cong {\left(4/N_{C}SN{R}_d\right)}^2$, where the desired signal-to-noise ratio, $SN{R}_d$, is typically 10 to 100. If we take $SN{R}_d=100$ and estimate $B_x\cong 30$ and N = 140 as for our test image, then this ratio is ${\left\{0.04/\left[{\left(140\right)}^2-{\left(30\right)}^2\right]\right\}}^2=4.57\times {10}^{-12}$. This amounts to 0.4 h versus ten million years!

7. Conclusions

This paper is a survey of a sequence of our papers [5,6,7,8] that the author has developed for an algorithm capable of two-dimensional imaging that enormously reduces the integration times for the HBT effect while retaining the advantages of simple, inexpensive flux collecting hardware, robustness to misalignments, immunity to seeing conditions, and unlimited baselines and image resolution. This significant step forward merits the name “Intensity Correlation Imaging” (which we designate as ICI hereafter).

Now, how may our readers say that the mathematics are rigorous and yet say that it lacks validation? Someone could give you reams of empirical data but that would not suffice because how would you know that the data are real? This manuscript establishes an algorithm that produces the reality of a noise-free image. The best way to validate an algorithm (and the only way) is to try it out. I would expect readers to copy (perhaps in a simple MatLab script) the algorithm as it is stated in the manuscript and then validate the algorithm by running it. Please think about this! Let us suppose that my readers see an Appendix A that has the MatLab script with a space for a nonzero-noise image that the user can select at will to run the simulation and validate the algorithm (and find improvements!). I believe that this is very helpful for the validation of the theory and for the continued growth of the theory and practice of the photonic community. Consequently, I have added Appendix A containing the complete algorithm.

Subsequent to the present paper, the practical benefits of ICI to astronomical science must be pursued, emphasizing its low cost and very high resolution. Already, Reference [17] has initiated a design for a ground-based ICI observatory for geostationary satellite inspection with (1) a resolution of 10 cm (at 36,000 km), (2) a range of apparent magnitudes of 10 through 14, (3) integration times from 0.043 min to 5.18 min, and (4) an integration time reduction of $\cong {10}^{-30}$ below the Brown–Twiss calculation. No conventional visible range multipixel imager with these parameters can claim a cost below USD 140 million. Furthermore, designs for high-resolution low-Earth-orbit space-debris-tracking as well as deep-space astronomical research facilities are being planned.

These efforts could open the door to the construction of advanced ICI observatories that are well within the financial resources of colleges and universities around the world yet able to compete with very large telescopes, such as those discussed in the National Academy of Sciences’ most recent decadal survey in astronomy and astrophysics [18]. We look forward to significant advances in astronomical technology as well as a broadening participation in scientific education.