Correlation Plenoptic Imaging: An Overview

Abstract

Plenoptic imaging (PI) enables refocusing, depth-of-field (DOF) extension and 3D visualization, thanks to its ability to reconstruct the path of light rays from the lens to the image. However, in state-of-the-art plenoptic devices, these advantages come at the expenses of the image resolution, which is always well above the diffraction limit defined by the lens numerical aperture (NA). To overcome this limitation, we have proposed exploiting the spatio-temporal correlations of light, and to modify the ghost imaging scheme by endowing it with plenoptic properties. This approach, named Correlation Plenoptic Imaging (CPI), enables pushing both resolution and DOF to the fundamental limit imposed by wave-optics. In this paper, we review the methods to perform CPI both with chaotic light and with entangled photon pairs. Both simulations and a proof-of-principle experimental demonstration of CPI will be presented.

1. Introduction

Plenoptic imaging is a recently established optical imaging technique, characterized by the possibility to simultaneously detect both the spatial distribution and the propagation direction of light in a given scene [1,2,3]. The ability to reconstruct light paths can be used, in post-processing, to refocus out-of-focus objects, change the point of view on the scene and extend the DOF. PI is also one of the simplest and fastest methods to obtain three-dimensional images with the current technology [4,5,6,7,8,9,10]; in particular, it is among the imaging methods able to achieve 3D imaging in a single shot [11,12,13,14,15]. PI does not require either multiple sensors for multiperspective acquisition, or scanning or interferometric methods; single-shot 3D imaging is instead accomplished by a simple modification of ordinary imaging devices (such as cameras and microscopes) that does not involve using multiple excitation beams [13], multifocus gratings combined with aberration corrections [14] or spatial light modulators [15]. The key component of state-of-the-art plenoptic cameras is a microlens array placed in front of the sensor to ensure that a twofold information is encoded in the intensity detected by each pixel: the distribution of light in the object plane, and the direction of light between the object and the imaging device [16,17,18,19,20]. In the first implemented configuration, the microlenses were inserted in the native image plane, and were employed to reproduce repeated images of the main camera lens on the sensor behind them [3,21]. Light rays could thus be traced by joining each image “point” (i.e., each microlens) with each pixel associated with the image of the camera lens. To this end, the camera lens is divided in $N_u$ portions per side; hence, each portion of the camera lens has linear size $D/N_u$, with D the lens diameter, and reproduces a ”sub-image” of the scene. Sub-images are clearly endowed with a larger DOF than the ordinary image reproduced by the whole camera lens; this explains the physical origin of the improved DOF of plenoptic imaging. In addition, the sub-image formed by each portion of the camera lens offers a different perspective on the scene of interest, as required for 3D imaging. The drawback of these intriguing peculiarities of standard PI is the loss of spatial resolution, which is now defined by the transverse size of the microlenses: resolution worsens by the same factor $N_u$ that fixes directional resolution. Hence, the potentials of plenoptic imaging are strongly limited by the insertion of the microlens array and the use of a single sensor for retrieving both spatial and directional information.

Despite its limitation, PI is currently employed in the most diverse applications, including 3D imaging and sensing [5,22], stereoscopy [2,23,24], particle image velocimetry [25], particle tracking and sizing [26], wavefront sensing [6,27,28,29], microscopy [4,6,11,30] and digital cameras with refocusing capabilities [31]. Plenoptic imaging has also been employed in surgical robotics [32], endoscopic applications [33], and blood-flow visualization [34].

In a more recent scheme, called Plenoptic 2.0, the microlenses create redundant images of portions of the scene of interest, in order to somewhat smoothen the compromise between loss of resolution and increased DOF [16,17,18,19]. Attempts to weaken the resolution vs. DOF trade-off have been made by using signal processing and deconvolution [4,6,35,36,37], and other algorithms and analysis tools have been developed [8,38].

In this perspective, we have recently proposed a fundamentally different approach to PI, named correlation plenoptic imaging (CPI), which exploits the spatio-temporal correlation properties of light beams to physically decouple the image formation from the retrieval of the directional information [39]. We perform “spatial” and “directional” measurements on two separate sensors: one gives the image of the scene of interest, while the other one gives the image of the focusing element responsible for image formation (e.g., the camera lens of standard PI). The correlation measurement combines such structured information to provide the same kind of information acquired by a conventional plenoptic camera, but without losing image resolution. In fact, the retrieval of the directional information is exactly the same as in standard PI, namely, it is obtained by joining “points” of the image of interest with “points” of the image of the focusing element.

We have theoretically shown that CPI can be achieved by exploiting the correlation properties of both chaotic light [39,40] and entangled photons from spontaneous parametric down-conversion (SPDC) [41]. We have also performed the first experimental proof of CPI with chaotic light [42], and we are currently generalizing the experimental demonstration to entangled photons. The results suggest that CPI can improve the power of plenoptic imaging, opening the way to promising applications, especially in fields like microscopy and 3D imaging where fast acquisition must be combined with high resolution. In addition, entangled photons from SPDC provide the possibility to perform CPI by correlating photons of different wavelengths in the two arms of the setup: light illuminating the object in one arm is not required to have the same wavelength as light remotely detected in the other arm [43,44,45]. This feature is interesting both in view of applications requiring specific illumination wavelengths for the object and to optimize the detection efficiency. Entanglement has the further advantage of enabling high signal-to-noise ratio (SNR) images at low photon fluxes [46,47,48], which is particularly interesting when radiation damage of the sample is an issue (e.g., in biomedical microscopy).

Unlike previous fundamental demonstrations [49,50,51], CPI exploits the simultaneous momentum and position correlation of light to address intrinsic limitations affecting practical imaging system, such as the resolution versus DOF compromise. Compared to other 3D imaging techniques based on the simultaneous detection of both the spatial distribution and the propagation direction of light, CPI does not require either delicate interferometric techniques, as in digital holographic microscopy [52], or phase retrieval algorithms, as in Fourier ptychography [53]. Furthermore, similar to standard PI, CPI does not require fast pulsed light, as Time-Of-Flight (TOF) imaging [26,54,55,56,57,58,59], and, compared to confocal microscopy [60], offers the advantage of being a scanning-free imaging modality.

In this paper, we review the main aspects of CPI, and present different schemes we have so far developed. We start, in Section 2, with the first proposed CPI setup, where an adequate modification of lensless ghost imaging is employed to retrieve the plenoptic ghost image of the object. In Section 3, we report the first experimental realization of this CPI scheme. In Section 4, we present an alternative CPI scheme, where the image of the object is directly focused on one sensor, while ghost imaging (i.e., correlation measurements) is employed to image the focusing element in order to gain the required directional information; the practical advantages of this scheme will also be discussed. In Section 5, we present CPI based on entangled photon pairs generated by SPDC.

2. Correlation Plenoptic Imaging with Chaotic Light—First Scheme

The general working principle of correlation plenoptic imaging can be understood by considering the first setup we have developed and experimentally demonstrated.

As reported in Figure 1, light emitted by a chaotic (in our case, pseudothermal) source is divided in two arms by a beam splitter (BS): the reflected beam propagates in arm a for a distance $z_a$ from the source before being detected by the sensor array ${\mathrm{D}}_a$; the transmitted beam travels in arm b, where light impinges on a transmissive object at a distance $z_b$ from the source, and is then collected by a lens $L_b$ (characterized by the focal length F) for reproducing on the sensor array ${\mathrm{D}}_b$ the image of the source. The fluctuations of the intensity detected by each pixel of the two sensor arrays are monitored in time to reconstruct their spatio-temporal correlations. By correlating the total intensity on sensor ${\mathrm{D}}_b$ with the one retrieved by each pixel of sensor ${\mathrm{D}}_a$, we obtain the “ghost image” (GI) of the object, which is perfectly focused when $z_a=z_b$ [61,62,63,64,65]; here, the chaotic source is playing the role of a focusing element, as shown by the unfolded representation (“Klyshko-like picture”) of Figure 2a [61,62]. The high-resolution detector array ${\mathrm{D}}_b$ is not required to perform ghost imaging, where a bucket detector with no spatial resolution behind the object is sufficient. However, in the present scheme, the detector array ${\mathrm{D}}_b$ is crucial: by reproducing the image of the light source, it enables joining points of the object with points of the source plane, thus giving information on the direction of light, as required to perform plenoptic imaging. Both spatial and angular information are encoded in the correlation of intensity fluctuations; hence, correlation measurements provide the necessary information for performing the typical tasks of plenoptic imaging, such as refocusing out-of-focus details of the 3D object of interest (i.e., details placed outside the DOF of the standard image, at $z_a\ne z_b$).

2.1. Correlation Functions in CPI

The correlation between the fluctuations $\Delta I_a$ and $\Delta I_b$ of the intensities $I_a$ and $I_b$, evaluated at the transverse coordinates ${\mathit{\rho }}_a$ and ${\mathit{\rho }}_b$ defined on the planes of detectors ${\mathrm{D}}_a$ and ${\mathrm{D}}_b$, respectively, reads

$$\mathsf{\Gamma }({\mathit{\rho }}_a,{\mathit{\rho }}_b)=\langle \Delta I_a({\mathit{\rho }}_a)\Delta I_b({\mathit{\rho }}_b)\rangle =\langle I_a({\mathit{\rho }}_a)I_b({\mathit{\rho }}_b)\rangle -\langle I_a({\mathit{\rho }}_a)I_b({\mathit{\rho }}_b)\rangle ;$$

(1)

the symbol $\langle \cdots \rangle$ denotes the expectation value on the state of the source, and can be determined by taking into account the source statistics. For the setup represented in Figure 1, the computation in the case of a chaotic and quasi-monochromatic source with central frequency ${\omega }_0$, yields (see Appendix A for details)

$${\mathsf{\Gamma }}_{(z_a,z_b)}({\mathit{\rho }}_a,{\mathit{\rho }}_b)={\left|\int d{\mathit{\rho }}_{o}d{\mathit{\rho }}_{s}A({\mathit{\rho }}_0)\mathcal{S}({\mathit{\rho }}_s)e^{i\frac{{\omega }_0}{c}\phi ({\mathit{\rho }}_o,{\mathit{\rho }}_s;{\mathit{\rho }}_a,{\mathit{\rho }}_b)}\right|}^2,$$

(2)

with

$$\phi ({\mathit{\rho }}_o,{\mathit{\rho }}_s;{\mathit{\rho }}_a,{\mathit{\rho }}_b)=\frac{{\mathit{\rho }}_s^2}{2}\left(\frac{1}{z_b}-\frac{1}{z_a}\right)-\frac{{\mathit{\rho }}_o}{z_b}·\left({\mathit{\rho }}_s+\frac{{\mathit{\rho }}_b}{M}\right)+\frac{{\mathit{\rho }}_s·{\mathit{\rho }}_a}{z_a},$$

(3)

where ${\mathit{\rho }}_o$ is the transverse coordinate on the plane of the sample, ${\mathit{\rho }}_s$ is the transverse coordinate on the source plane, c is the speed of light, ${\omega }_0$ is the pump frequency of the laser pump, A is the aperture function of the object, $\mathcal{S}$ is the transverse intensity profile of the source and M the absolute magnification of the image of the source on ${\mathrm{D}}_b$. The dependence of $\mathsf{\Gamma }$ on the distances $z_a$ and $z_b$ has been explicitly highlighted to enable easily checking whether focused or out-of-focus images are being considered.

Integration of $\mathsf{\Gamma }$ over the whole directional sensor ${\mathrm{D}}_b$ yields an incoherent image of the object, whose point-spread function (PSF) is related with the Fourier transform of the function $\mathcal{S}({\mathit{\rho }}_s)e^{i\frac{{\omega }_0}{2c}(z_b^{-1}-z_a^{-1}){\mathit{\rho }}_s^2},$ which we shall indicate by $\tilde{\mathcal{S}}$. The minimal point-spread thus occurs at $z_b=z_a$, where $\tilde{\mathcal{S}}$ coincides with the Fourier transform of $\mathcal{S}$; this is the typical PSF of ghost imaging with chaotic light. In particular, the focused ghost image reads

$${\mathsf{\Sigma }}^{\mathrm{foc}}({\mathit{\rho }}_a)=\int d{\mathit{\rho }}_b{\mathsf{\Gamma }}_{(z_a,z_a)}({\mathit{\rho }}_a,{\mathit{\rho }}_b)\propto \int d{\mathit{\rho }}_o{|A({\mathit{\rho }}_o)|}^2{\left|\tilde{\mathcal{S}}\left[\frac{{\omega }_0}{c{z}_a}\left({\mathit{\rho }}_o-{\mathit{\rho }}_a\right)\right]\right|}^2,$$

(4)

which entails a quasi one-to-one correspondence between points of the object (${\mathit{\rho }}_o$) and pixels of the sensor ${\mathrm{D}}_a$ (${\mathit{\rho }}_a$), with an uncertainty $\Delta {\rho }_a={\lambda }_0{z}_a/D_s$ defined by the effective diameter $D_s$ of the source. On the other hand, sensor ${\mathrm{D}}_b$ collects the image of the source as given by Equation (2), thus enabling a correspondence between its pixels (${\mathit{\rho }}_b$) and points of the source plane (${\mathit{\rho }}_b=-M{\mathit{\rho }}_s$), with an uncertainty $\Delta {\rho }_b=M{\lambda }_0{z}_b/a$ defined by the typical size a of the smallest detail of the object. The Klyshko-like pictures depicted in Figure 2 enables a comparison between the focusing effect of the source in lensless ghost imaging (a) [66] and correlation plenoptic imaging (b). In fact, the structure of the nontrivial term in Equation (2) indicates that, in a Klyshko-like unfolded setup, the source acts as a phase conjugate mirror, and the correlated modes in the two arms of the setup are characterized by similar transverse momenta.

2.2. Point-Spread Function and Plenoptic Properties

To better understand the physical meaning of the point-spread-function of CPI, and its consequences on the DOF enhancement, let us consider a light source with a Gaussian intensity profile

$$\mathcal{S}({\mathit{\rho }}_s)=\frac{1}{2\pi {\sigma }^2}exp\left(-\frac{{\mathit{\rho }}_s^2}{2{\sigma }^2}\right).$$

(5)

In this case, the coherent PSF in Equation (2) reduces to

$$\mathcal{C}({\mathit{\rho }}_o-\alpha {\rho }_a)=\int d{\mathit{\rho }}_s\mathcal{S}({\mathit{\rho }}_s)e^{\frac{{\omega }_0}{c{z}_b}\frac{{\mathit{\rho }}_s^2}{2}(1-\alpha )}{e}^{-\frac{i{\omega }_0}{c{z}_b}\left({\mathit{\rho }}_o-\alpha {\mathit{\rho }}_a\right)·{\mathit{\rho }}_s}\propto exp\left(-\frac{1}{2}{\left(\frac{{\omega }_0\sigma }{c{z}_b}\right)}^2\frac{|{\mathit{\rho }}_0-\alpha {\mathit{\rho }}_a{|}^2}{1-\frac{i{\omega }_0{\sigma }^2}{c{z}_b}(1-\alpha )}\right),$$

(6)

with $\alpha =z_b/z_a$. The PSF of the incoherent image, as obtained by integrating Equation (2) over ${\mathit{\rho }}_b$, comes out to be the square modulus of the result in Equation (6), namely,

$$\mathcal{J}({\mathit{\rho }}_o-\alpha {\mathit{\rho }}_a)\propto exp\left(-{\left(\frac{{\omega }_0\sigma }{c{z}_b}\right)}^2\frac{|{\mathit{\rho }}_0-\alpha {\mathit{\rho }}_a{|}^2}{1+{\left(\frac{{\omega }_0{\sigma }^2}{c{z}_b}(1-\alpha )\right)}^2}\right).$$

(7)

The physics behind the wider depth of field of CPI with respect to standard ghost imaging comes out from the comparison of Equations (6) and (7). In fact, in geometrical optics (${\omega }_0\to \infty$), the variance of the incoherent PSF of Equation (7), typical of ghost imaging, approaches $\sigma |1-\alpha |$ and is thus independent of frequency. In the same regime, the coherent PSF of Equation (6), associated with correlation plenoptic imaging, reduces to an imaginary quadratic exponential, with typical width $\sqrt{2\pi {\lambda }_0{z}_b|1-\alpha |}$. Therefore, the width of the coherent PSF vanishes in the geometrical optics approximation. Most important, the dependence of the two PSFs on defocusing ($|1-\alpha |$) is such that the (imaginary) width of the coherent PSF scales much slower than the width of the incoherent PSF. This is the reason behind the wider DOF of (coherent) correlation plenoptic imaging, as compared to inchoerent imaging, whether standard or ghost. Now, to unveil the plenoptic properties encoded in ${\mathsf{\Gamma }}_{(z_a,z_b)}$, it is worth considering the limit of Equation (2) in the short-wavelength regime, when the integral is dominated by the stationary points of the phase. In particular, stationarity with respect to ${\mathit{\rho }}_o$ yields

$${\mathit{\rho }}_s+\frac{{\mathit{\rho }}_b}{M}=0,$$

(8)

namely, the geometrical correspondence between points of the source and points of the sensor ${\mathrm{D}}_b$, with magnification M and inversion of the image, as already discussed. The stationarity condition with respect to ${\mathit{\rho }}_s$ yields a less trivial result, that, combined with Equation (8), identifies the object point ${\mathit{\rho }}_o$ that provides the dominant contribution to the integral, at the given detection positions ${\mathit{\rho }}_a$ and ${\mathit{\rho }}_b$, which is:

$${\mathit{\rho }}_o=\frac{z_b}{z_a}{\mathit{\rho }}_a-\frac{{\mathit{\rho }}_b}{M}\left(1-\frac{z_b}{z_a}\right).$$

(9)

Hence, in the geometrical-optics limit, the intensity correlation function reads

$${\mathsf{\Gamma }}_{(z_a,z_b)}({\mathit{\rho }}_a,{\mathit{\rho }}_b)\sim \mathcal{S}{\left(-\frac{{\mathit{\rho }}_b}{M}\right)}^2{\left|A\left[\frac{z_b}{z_a}{\mathit{\rho }}_a-\frac{{\mathit{\rho }}_b}{M}\left(1-\frac{z_b}{z_a}\right)\right]\right|}^2,$$

(10)

namely, it reproduces the magnified and displaced image of the object (A), multiplied by the source intensity profile, as shown in Figure 3a. When $z_b\ne z_a$, integration over ${\rho }_b$ blurres the image of the aperture function of the object, thus giving an out of focus ghost image, as shown in Figure 3c. This indicates the crucial role played in CPI by the high-resolution detector ${\mathrm{D}}_b$, as opposed to the bucket detector of ghost imaging. In fact, magnified and displaced images obtained by the correlation measurements can be reshaped and realigned (i.e., refocused) by employing the following scaling property,

$${\mathsf{\Gamma }}_{(z_a,z_b)}\left[\frac{z_a}{z_b}{\mathit{\rho }}_a-\frac{{\mathit{\rho }}_b}{M}\left(1-\frac{z_a}{z_b}\right),{\mathit{\rho }}_b\right]\sim \mathcal{S}{\left(-\frac{{\mathit{\rho }}_b}{M}\right)}^2{\left|A({\mathit{\rho }}_a)\right|}^2,$$

(11)

as shown in Figure 3b. The independence of the image ${|A|}^2$ of ${\mathit{\rho }}_b$ guarantees that integration over ${\mathit{\rho }}_b$ does not compromise the image quality. In fact, as shown in Figure 3d, the final image

$${\mathsf{\Sigma }}_{(z_a,z_b)}^{\mathrm{ref}}({\mathit{\rho }}_a)=\int d{\mathit{\rho }}_b{\mathsf{\Gamma }}_{(z_a,z_b)}\left[\frac{z_a}{z_b}{\mathit{\rho }}_a-\frac{{\mathit{\rho }}_b}{M}\left(1-\frac{z_a}{z_b}\right),{\mathit{\rho }}_b\right]\sim {\mathsf{\Sigma }}^{\mathrm{foc}}({\mathit{\rho }}_a)$$

(12)

now approximates, up to an intensity rescaling, the focused incoherent image in Equation (4). The refocusing alghoritm of Equation (11) is formally identical to the one employed in standard plenoptic imaging [3]. In fact, in analogy with standard plenoptic imaging, the key for refocusing is the information on the direction of light propagating between the object and the focusing element. In our CPI scheme, by measuring correlations, we can reconstruct the path of light rays from the source to the object, as clarified in Figure 2b. Hence, once again, we find that the coherent images associated with any fixed value of ${\mathit{\rho }}_b$, as described by Equations (10) and (11), have a wider depth of field with respect to the incoherent images obtained by a mere integration over ${\mathit{\rho }}_b$. However, such images are formed by a very small fraction of light propagating through the object, and are thus affected by a low signal-to-noise ratio. The SNR highly improves when summing all the rescaled images of Equation (11) over ${\mathrm{D}}_b$, to get the final refocused image of Equation (12).

Finally, both the result in Equation (10) and the plot in Figure 3a indicate that, when the image is out of focus, each pixel on the sensor ${\mathrm{D}}_b$ represents a different point of view from which the image of the object is projected onto the sensor ${\mathrm{D}}_a$. Hence, imaging the light source on the sensor ${\mathrm{D}}_b$, enables acquiring multi-perspective images of the scene of interest.

The change of viewpoint is a common feature of both PI and CPI, with the only difference that, in CPI, it is obtained with a single lens ($L_b$) rather than by the microlens array of standard PI; therefore, potentially, it is significantly larger in CPI than in standard PI. Despite not using it in the present work, it is worth emphasizing the key role played by the wide change of perspective and its achievable resolution, for implementing 3D imaging.

Based on Equations (11) and (12), to get a refocused image, one should know with sufficient precision both the source-to-sensor distance $z_a$ and the source-to-object distance $z_b$. In reality, while $z_a$ is determined by experimental settings, $z_b$ is generally unknown. In this case, the refocusing algorithm must be applied for different values of $z_b$, until the optimal sharpness of the image is found. This is shown in Figure 4, where we present a simulation of different images of an out of-focus object [67]: (a) the blurred ghost image; (b) the refocused CPI image; (c) and (d) two refocusing attempts with the correct value of $z_a$ and an incorrect value of $z_b$. The last two images are evidently less sharp than the one in (b). This also indicates that the refocusing capability of CPI can also be exploited for measuring distances.

2.3. Depth-of-Field Improvement

To analyze the fundamental limits defining the maximum achievable resolution and DOF of CPI, we shall compare the DOF of CPI and ghost imaging, which is in fact equivalent to any standard imaging (SI) obtained by a focusing element having the same numerical aperture as the chaotic source. For simplicity, we consider a 1D object $A({\mathit{\rho }}_o)=A(x_0)$ and a source with a Gaussian profile, as in Equation (5). In Figure 5 and Figure 6, we report the incoherent ghost image (left column)

$${\mathsf{\Sigma }}_{(z_a,z_b)}\left(\frac{z_a}{z_b}{x}_a\right)=\int d{x}_o{|A(x_o)|}^2{\mathcal{J}}_1(x_o-x_a),$$

(13)

which, for $z_a=z_b$, coincides with ${\mathsf{\Sigma }}^{\mathrm{foc}}$ of Equation (4), the coherent image from CPI (central column)

$${\mathsf{\Gamma }}_{(z_a,z_b)}\left(\frac{z_a}{z_b}{x}_a,x_b=0\right)={\left|\int d{x}_{o}A(x_o){\mathcal{C}}_1(x_o-x_a)\right|}^2$$

(14)

and the refocused image from CPI (right column)

$${\mathsf{\Sigma }}_{(z_a,z_b)}^{\mathrm{ref}}\left(x_a\right)=\int d{x}_b{\mathsf{\Gamma }}_{(z_a,z_b)}\left(\frac{z_a}{z_b}{x}_a-\frac{x_b}{M}\left(1-\frac{z_a}{z_b}\right),x_b\right),$$

(15)

with ${\mathcal{J}}_1(x)=\int dy\mathcal{J}(x,y)$ and ${\mathcal{C}}_1(x)=\int dy\mathcal{C}(x,y)$, as defined in Equations (6) and (7). Single slits of different width a are considered in Figure 5, and a double-slit mask in Figure 6. The Gaussian profile of the source has width $\sigma =1.08\mathrm{mm}$. In line with the experiment that will be discussed in the following section, we fix the distance $z_a$ between the source and the sensor ${\mathrm{D}}_a$, while changing the source-to-object distance $z_b$. Comparison of panels (b)–(e)–(h) with panels (c)–(f)–(i) of Figure 5 indicates that the maximum achievable DOF in CPI is limited by diffraction at the object, which hinders the detection of directional information, and hence the ability to refocus. Such dependence can be understood in terms of Klyshko picture [68], as applied to ghost imaging with chaotic light [62]. In addition, Figure 6 shows that interference effects, typical of coherent imaging, also limit the maximum achievable DOF of the refocused image. On the other hand, the resolution of the focused image is only limited by the size of the focusing element. CPI thus reaches the fundamental limits imposed by the wave nature of light to both image resolution and DOF.

Correlation Plenoptic Imaging offers the unique opportunity to refocus without sacrificing diffraction-limited image resolution, as defined by the numerical aperture of the imaging system. In Figure 7c, the dashed line represents the estimate, based on geometrical optics, for the maximum range of “perfect” refocusing in CPI [39], which is obtained from the condition

$$\left|1-\frac{z_a}{z_b}\right|<\frac{\Delta x}{\Delta u}=\frac{d{z}_a/z_b}{max[\lambda z_b/a,2\lambda /(M_b{\mathrm{NA}}_b),2{\delta }_u/M]},$$

(16)

with $\Delta x$ the resolution on the spatial sensor ${\mathrm{D}}_a$ and $\Delta u$ the resolution on the source plane. On the right-hand side of Equation (16), we have considered d as the distance between object points that we want to resolve (e.g., the centers of a double slit), while a is the typical size of the smallest details of the object (e.g., the slit width). The resolution limit $\Delta x=d{z}_a/z_b$ is defined by the geometrical projection of the image on the sensor plane. The resolution limit $\Delta u$ is defined by the largest contribution among the ones determined by diffraction at the object (i.e., $\lambda z_b/a$), the numerical aperture of $L_b$, or the pixel size ${\delta }_u$; the last two contributions enter into play only for objects very close to the light source (i.e., for $z_b=2a/(M_b{\mathrm{NA}}_b)$ and $z_b=2{\delta }_{u}a/(M\lambda )$, respectively). Hence, the physical quantities that generally define the directional and spatial resolution of correlation plenoptic imaging are the source-to-object distance $z_b$ and the quantities a and d that characterize the object transmission function. Figure 7c shows, in a density plot, the visibility of the images of double slits with center-to-center distance d and width $a=d/2$, evaluated in the experimental setting described in Section 3. The plot tests the reliability of the geometrical prediction in Equation (16), and also reveals the real (wave-optics) limits of resolution and depth of field in CPI. Oscillations that can be observed in the density plot are due to the coherent nature of correlation plenoptic imaging.

To compare CPI with both standard imaging and standard PI, we consider imaging devices having the same NA as the light source in our scheme, and report in Figure 7a,b the visibility they achieve. In such cases, the quantity $z_b-z_a$ is the difference between the actual position of the object and the position of the plane focused on the detector by the imaging lens. We have considered the case $N_u=3$ (namely, $3\times 3$ resolution cells on the focusing element) in the case of standard PI, in order to gain directional information without hindering too much spatial resolution. Comparing the three plots in Figure 7, we can observe that CPI combines in an optimal way the advantages of standard imaging and plenoptic imaging, since diffraction-limited resolution is preserved and DOF is increased beyond the typical values achieved in standard PI. Moreover, the smaller object details that can be refocused at $z_b=z_a/2$ ($d=\sqrt{8\lambda z_a}\simeq 2.8\Delta x^f$) can always be refocused for close up $z_b<z_a$, no matter how misfocused they are. For $z_b>z_a$, the DOF associated with CPI is significantly larger than the ones characterizing standard imaging and standard PI.

It is worth emphasizing that the DOF of the standard image represents the axial resolution of CPI ($\Delta z^{CPI}=\lambda /{\mathrm{NA}}^2$); hence, the ratio between the depth of fields of CPI and standard imaging provides an estimate of the number of unique planes that can be refocused by CPI.

3. Experimental Demonstration of CPI

The experimental setup employed to demonstrate CPI is represented in Figure 10. The light source is a continuous-wave single-mode laser with wavelength $\lambda =532\mathrm{nm}$ and tunable power up to $5\mathrm{W}$ (Azur Light Systems ALS-532nm-SF, Azur Light Systems, Pessac, France). To obtain a controllable chaotic source, the laser beam is expanded to a spot size $\sigma =1.08\mathrm{mm}$, is passed through a polarizer and impinges on a rotating ground glass disk, spinning at $0.05\mathrm{Hz}$, at a distance of about $4\mathrm{cm}$ from its center. As depicted in Figure 10, light is then divided by a polarizing beam-splitter (PBS). The combination of polarizer and PBS enables optimizing the SNR by balancing the intensities at the sensors ${\mathrm{D}}_a$ and ${\mathrm{D}}_b$, which are different areas of the same scientific complementary metal-oxide semiconductor (sCMOS) sensor (Hamamatsu ORCA-Flash 2.8 camera C11440-10C, Hamamatsu Corporation, Shizuoka Prefecture, Hamamatsu, Japan). The reflected beam passes through the object of interest (ThorLabs 1951 USAF Resolution Test Targets, Newton, NJ, USA), propagates toward a lens ($L_b$) of focal length $F=300\mathrm{mm}$, and reaches the angular sensor ${\mathrm{D}}_b$, which lies in the conjugate plane of the source (image has unitary magnification $M=1$). The transmitted beam propagates toward a lens ($L_a$) of focal length $f_a=125\mathrm{mm}$, that reproduces on the spatial sensor ${\mathrm{D}}_a$ the image of the “ghost-imaging plane”, set at a distance $z_a=92\mathrm{mm}$ from the source, with a magnification $M_a=1$. In our case, the resolution at focus is $\Delta x^f=\lambda /\mathrm{NA}=14\mathsf{\mu }\mathrm{m}$, where NA is the smaller numerical aperture between the ones of the source and of $L_a$. For objects at the resolution limit, $\mathrm{DOF}=\lambda /{\mathrm{NA}}^2=0.37\mathrm{mm}$. The camera is characterized by a pixel size of $3.6\mathsf{\mu }\mathrm{m}$, which is much smaller than both the spatial and the directional resolution. We thus perform a binning of the camera pixels to match the effective pixels ${\delta }_x$ and ${\delta }_u$ with the spatial and directional resolutions, respectively. In particular, during data acquisition, we perform a 2 × 2 binning to get ${\delta }_x=7.2\mathsf{\mu }\mathrm{m}\approx \Delta x^f/2$. In post-processing, a further 10 × 10 binning is performed on the region of the camera sensor dedicated to the angular measurement, thus getting ${\delta }_u=72\mathsf{\mu }\mathrm{m}<\Delta u/2$, with $\Delta u=\lambda z_b/a$. The latter is determined, for the chosen values of $z_b$ and object size, by diffraction at the object. The test target mimics small details and allows for easily monitoring the image resolution, both in the out-of-focus and in the refocused image. We acquire 50,000 frames for all measurements at a frame rate of $45.4{\mathrm{s}}^{-1}$, with an exposure time ${\tau }_{\mathrm{meas}}=21\mathsf{\mu }\mathrm{s}$ approximately 100 times smaller than the chaotic source coherence time. The acquired frames are processed to evaluate the spatio-temporal intensity correlation, which is expected to converge to Equation (2). The CPI images are reported in Figure 8 and Figure 9a,b. In Figure 8, we report the experimental results obtained for element 3 of group 2 of the test target: the three slits have center-to-center distance $d=0.198\mathrm{mm}$ and slit width $a=d/2$ (measurement B in Figure 7). In the left column, we report the out-of-focus image obtained on ${\mathrm{D}}_a$ by measuring correlation with the whole detector ${\mathrm{D}}_b$, when the mask is placed significantly out of focus ($z_b-z_a\gtrsim 20\mathrm{mm}$); this is equivalent to the blurred image that any conventional imaging system, characterized by the same NA as our CPI scheme, would retrieve with the same defocusing. In the right column, we report the same image after implementing the CPI refocusing algorithm in Equation (12). In Figure 9, we report the experimental refocused images obtained, respectively, for element 3 of group 2 (having $d=0.198\mathrm{mm}$) placed at $z_b-z_a=-46\mathrm{mm}$ (measurement A in Figure 7), and for element 4 of group 1 (having $d=0.354\mathrm{mm}$) placed at $z_b-z_a=41\mathrm{mm}$ (measurement C in Figure 7). The corresponding standard ghost images are much more blurred than the one reported in Figure 8a, and therefore have not been reported. The SNR in Figure 9a is lower than in Figure 8b and Figure 9b because the displaced coherent images retrieved for $z_b=z_a/2$ are two times larger than the object, and distributed over a region wider than the illuminated area; the coherent images are thus affected by a poor SNR, which reflects on the final refocused image. To avoid this issue, the divergence of the light source should be designed to account for such displacement and enlargement.

The experimental parameters used to obtain the results in Figure 8 and Figure 9 are identified by points A, B and C in Figure 7. In all three experiments, correlation plenoptic imaging provides a relevant DOF advantage: CPI enables refocusing the object associated with A and B in a 7 times larger range than in standard imaging, and in a $2.5$ times larger range than with standard PI having $N_u=3$. For the object associated with point C, the refocusing range of CPI is four times larger than the DOF of standard imaging and two times larger than the DOF of a standard PI device characterized by a spatial resolution that is three times worse (i.e., $N_u=3$).

4. A Different Architecture of CPI with Chaotic Light

We are now going to discuss the possibility of performing correlation plenoptic imaging in a setup that also enables standard imaging, namely, where the image of the object is available from ordinary intensity measurements. The advantage is the possibility to directly monitor the object while plenoptic imaging is performed. The proposed setup is represented in Figure 11; it is characterized by the same components as the scheme discussed in Section 2, but the light source and the lens now play opposite roles. Actually, the lens $L_b$ now focuses the object on detector ${\mathrm{D}}_b$, while the chaotic source enables for reproducing the ghost image of the lens by intensity correlation measurements [69]. For the ghost image of the lens to be focused on ${\mathrm{D}}_b$, we set

$$z_a=z_b+S_1.$$

(17)

The choice of imaging the lens $L_b$ is motivated by the need to gain directional information on light propagating from the object to ${\mathrm{D}}_b$. In fact, in the present scenario, $L_b$ is the focusing element, responsible for reproducing the image of the object of interest. The object is focused on ${\mathrm{D}}_b$ when the lens-to-sensor distance is $S_2=S_2^f$, with

$$\frac{1}{S_1}+\frac{1}{S_2^f}=\frac{1}{f};$$

(18)

however, since we are interested in demonstrating the refocusing power of this scheme, we shall not fix the value of $S_2$.

The correlation function is determined by an integral involving the source intensity profile $\mathcal{S}({\mathit{\rho }}_s)$, the object aperture function $A({\mathit{\rho }}_o)$ and the lens pupil function $P({\mathit{\rho }}_{\ell })$. However, in line with realistic experimental conditions, we shall assume that the source is large enough not to affect propagation of light in arm b of the setup; in this case, the value of $\mathsf{\Gamma }$ is determined with good approximation by the object features and the lens aperture, namely (up to irrelevant constants),

$${\mathsf{\Gamma }}_{(S_1,S_2)}({\mathit{\rho }}_a,{\mathit{\rho }}_b)={\left|\int d{\mathit{\rho }}_{o}d{\mathit{\rho }}_{\ell }A({\mathit{\rho }}_o)P({\mathit{\rho }}_{\ell })e^{i\frac{{\omega }_0}{c}\psi ({\mathit{\rho }}_o,{\mathit{\rho }}_{\ell };{\mathit{\rho }}_a,{\mathit{\rho }}_b)}\right|}^2,$$

(19)

where

$$\psi ({\mathit{\rho }}_o,{\mathit{\rho }}_{\ell };{\mathit{\rho }}_a,{\mathit{\rho }}_b)=\left(\frac{1}{S_2}-\frac{1}{S_2^f}\right)\frac{{\mathit{\rho }}_{\ell }^2}{2}-\left(\frac{{\mathit{\rho }}_o}{S_1}+\frac{{\mathit{\rho }}_b}{S_2}\right)·{\mathit{\rho }}_{\ell }+\frac{{\mathit{\rho }}_o·{\mathit{\rho }}_a}{S_1}.$$

(20)

With respect to the result obtained for the scheme of Figure 1, the source intensity profile is replaced here by the pupil function of the lens, and the focusing condition $z_a=z_b$ is replaced by $S_2=S_2^f$.

Similar to the previous scheme, the incoherent image of the object is recovered by integrating the correlation function over the whole angular detector ${\mathrm{D}}_a$

$${\mathsf{\Sigma }}^{\mathrm{foc}}({\mathit{\rho }}_b)=\int d{\mathit{\rho }}_a{\mathsf{\Gamma }}_{(S_1,S_2)}({\mathit{\rho }}_a,{\mathit{\rho }}_b)\propto \int d{\mathit{\rho }}_o{\left|A({\mathit{\rho }}_o)\right|}^2{\left|\int d^2{\mathit{\rho }}_{\ell }P({\mathit{\rho }}_{\ell })e^{-i\frac{{\omega }_0}{c}\left(\frac{{\mathit{\rho }}_o}{S_1}+\frac{{\mathit{\rho }}_b}{S_2^f}\right)·{\mathit{\rho }}_{\ell }}\right|}^2,$$

(21)

where the focusing condition is related with the disappearance of the quadratic phase in Equation (19). As expected, the width of the PSF is determined by the lens diameter ($D_{\ell }$), which is, $\Delta {\rho }_b\sim {\lambda }_0{S}_2^f/D_{\ell }$; hence, the object resolution is $\Delta {\rho }_o\sim {\lambda }_0{S}_1/D_{\ell }$. On the other hand, the incoherent ghost image of the lens is retrieved by integrating the correlation function over ${\rho }_a$, to get

$$\int d{\mathit{\rho }}_b{\mathsf{\Gamma }}_{(S_1,S_2)}({\mathit{\rho }}_a,{\mathit{\rho }}_b)\propto \int d{\mathit{\rho }}_{\ell }{\left|P({\mathit{\rho }}_{\ell })\right|}^2{\left|\tilde{A}\left(\frac{{\omega }_0}{c{S}_1}({\mathit{\rho }}_{\ell }-{\mathit{\rho }}_a)\right)\right|}^2.$$

(22)

The PSF of this image is represented by the Fourier transform of the object transmission function $\tilde{A}$; the resolution $\Delta {\rho }_a\sim {\lambda }_0{S}_1/a$ is thus fixed by the typical size a of the smallest details of the object. In addition, in this case, the angular resolution is thus only hindered by diffraction at the object, provided if the source is large enough not to play the role of a limiting pupil.

The plenoptic properties of the correlation function can be unveiled by imposing the stationary phase condition in the integral of Equation (19). The most relevant term in the geometrical optics regime reads

$${\mathsf{\Gamma }}_{(S_1,S_2)}({\mathit{\rho }}_a,{\mathit{\rho }}_b)\sim {\left|P({\mathit{\rho }}_a)\right|}^2{\left|A\left[\frac{S_1}{S_2}\left(-{\mathit{\rho }}_b+\left(1-\frac{S_2}{S_2^f}\right){\mathit{\rho }}_a\right)\right]\right|}^2.$$

(23)

As in the previous setup, the dependence of the retrieved correlation function on the coordinates (${\mathit{\rho }}_a\simeq {\mathit{\rho }}_{\ell }$) of the focusing element provides different viewpoints on the object, as required for 3D imaging. In addition, when the sensor ${\mathrm{D}}_b$ retrieves an out-of-focus image ($S_2\ne S_2^f$), refocusing can be achieved through a simple rescaling procedure, and the final image is given by

$${\mathsf{\Sigma }}_{(S_1,S_2)}^{\mathrm{ref}}({\mathit{\rho }}_b)=\int d{\mathit{\rho }}_a\mathsf{\Gamma }\left[{\mathit{\rho }}_a,\frac{S_1}{S_2^f}{\mathit{\rho }}_b+\left(\frac{S_1}{S_2^f}\right){\mathit{\rho }}_a\right]\simeq {\left|A\left(-\frac{S_1}{S_2^f}{\mathit{\rho }}_a\right)\right|}^2\sim {\mathsf{\Sigma }}^{\mathrm{foc}}({\mathit{\rho }}_b),$$

(24)

which approximates the focused image with an accuracy that improves as the geometrical-optics limit is approached.

Comparison between the Two Schemes

The present CPI setup, where the object is directly imaged by a lens $L_b$, ensures a larger control on the image resolution, which is now defined by the lens diameter rather than by the intensity profile of the chaotic source, as in the previous scheme. Another advantage of the new scheme is that the correlation function $\mathsf{\Gamma }$ depends on the square of the lens pupil function $P({\mathit{\rho }}_{\ell })$, which is usually a binary object; on the contrary, in the previous case, the dependence on the squared intensity profile of the source leads to a reduction of the signal in correspondence of dimmer areas. Still, this alternative scheme requires a source that is large enough as not to affect the resolution of the ghost image of the lens, which is thus only limited by diffraction at the object. An interesting follow-up of this research is the extension of the discussed methods to microscopy, one of the most intriguing applications of plenoptic imaging.

5. CPI with Entangled Photons

Up to now, we have discussed CPI setups based on chaotic light intensity correlations. Actually, the scheme can be generalized to any kind of correlated beam, provided that their properties are correctly taken into account. An outstanding example is given by entangled photon pairs, as produced by SPDC. Here, we will discuss the setup reported in Figure 12. In view of plenoptic imaging, this setup must enable the parallel acquisition of several images of the given scene, one for each propagation direction of light. We shall soon demonstrate that, also in this case, the sensor ${\mathrm{D}}_a$ retrieves N coherent ghost images of the object by means of correlation measurements with the N pixels of ${\mathrm{D}}_b$, each one giving a different viewpoint on the desired scene. This is quite intuitive if one considers that, similar to the first CPI setup in Figure 1, ${\mathrm{D}}_b$ reproduces the image of the light source. Like both previous CPI schemes, the lens $L_b$ alone replaces the microlens array required in standard plenoptic imaging.

As in the case of chaotic light, the intensity correlation measurement and, in the photon-counting regime, the coincidence detection, are described by the second order Glauber correlation function (see Equation (A1)). The expectation value is now taken over the two-photon signal-idler state produced by SPDC [70,71,72]

$$|\mathsf{\Psi }\rangle =\mathcal{N}\int d\nu s(LD\nu )\int d{\mathit{\kappa }}_{i}d{\mathit{\kappa }}_s{h}_{\mathrm{tr}}({\mathit{\kappa }}_i+{\mathit{\kappa }}_s)a_{{\mathit{k}}_i}^{†}{a}_{{\mathit{k}}_s}^{†}|0\rangle ,$$

(25)

where $\nu$ is the detuning of the signal and idler beams compared to their central frequencies ${\mathsf{\Omega }}_s={\mathsf{\Omega }}_i={\omega }_p/2$, related with the pump laser frequency ${\omega }_p$ by the phase-matching conditions, L is the longitudinal size of the SPDC crystal, D is the group velocity difference between the two beams, the function $s(LD\nu )$ is the SPDC biphoton spectrum [73,74], $\mathcal{N}$ is an irrelevant normalization, and $h_{\mathrm{tr}}$ is the Fourier transform of the transverse amplitude profile of the pump laser:

$$\mathcal{F}(\mathit{\rho })=\int d\mathit{\kappa }{e}^{i\mathit{\kappa }·\mathit{\rho }}{h}_{\mathrm{tr}}(\mathit{\kappa })$$

(26)

and the $a_{k_{i,s}}^{+}$’s are the operators that create photons with a definite momentum out of the vacuum $|0\rangle$. The computation of the second order Glauber correlation function, reported in the Appendix B, provides the following result:

$${\mathsf{\Gamma }}_{(z_a^{\prime },z_b)}({\mathit{\rho }}_a,{\mathit{\rho }}_b)\propto {\left|\int d{\mathit{\rho }}_{o}A({\mathit{\rho }}_o)\int d{\mathit{\rho }}_s\mathcal{F}({\mathit{\rho }}_s)e^{i\frac{\mathsf{\Omega }}{c}\phi ({\mathit{\rho }}_o,{\mathit{\rho }}_s;{\mathit{\rho }}_a,{\mathit{\rho }}_b)}\right|}^2,$$

(27)

with

$$\phi ({\mathit{\rho }}_o,{\mathit{\rho }}_s;{\mathit{\rho }}_a,{\mathit{\rho }}_b)=\left[\frac{1}{z_b}+\frac{1}{z_a}\left(1-\frac{\zeta (z_a,z_a^{\prime })}{z_a}\right)\right]\frac{|{\mathit{\rho }}_s{|}^2}{2}-\frac{\zeta (z_a,z_a^{\prime })}{z_a{z}_a^{\prime }}{\mathit{\rho }}_s·{\mathit{\rho }}_a-\frac{1}{z_b}\left({\mathit{\rho }}_s+\frac{{\mathit{\rho }}_b}{M}\right)·{\mathit{\rho }}_o,$$

(28)

where $\zeta (z_a,z_a^{\prime })=(z_{bF}+z_a)z_a/z_{bF}$.

5.1. Plenoptic Properties of the Correlation Function

The typical refocusing capability of PI characterized also the the CPI protocol due to the spatial and angular information encoded in the correlation function of Equation (27). To explicitly see this point, let us start by considering the simple case of a focused image, as obtained when the distance between the object and the source $z_b=z_{bF}$ satisfies the two-photon thin lens equation [61,75]

$$\frac{1}{z_a+z_{bF}}+\frac{1}{z_a^{\prime }}=\frac{1}{f}.$$

(29)

In this situation, the correlation function of Equation (27) gives the (incoherent) ghost image of the object, upon integration over the detection plane of ${\mathrm{D}}_b$ [61,75]:

$${\mathsf{\Sigma }}^{\mathrm{foc}}({\mathit{\rho }}_a)=\int d{\mathit{\rho }}_b{\mathsf{\Gamma }}_{(z_a,z_{bF})}({\mathit{\rho }}_a,{\mathit{\rho }}_b)\propto \int d{\mathit{\rho }}_o{|A({\mathit{\rho }}_o)|}^2{\left|h_{\mathrm{tr}}\left[\frac{\mathsf{\Omega }}{c{z}_{bF}}\left({\mathit{\rho }}_o+\frac{{\mathit{\rho }}_a}{m}\right)\right]\right|}^2,$$

(30)

where $m=z_a^{\prime }/(z_a+z_{bF})$ is the image magnification. This result holds if $h_{\mathrm{tr}}$ is more peaked around the origin than the Fourier transform of the imaging lens $L_a$; otherwise, it would be affected by the finite size of the lens. Such image is formally equivalent to the incoherent image obtained in an ordinary imaging system, were the PSF $h_{\mathrm{tr}}$ is given by the Fourier transform of the imaging lens pupil. However, as in the case of chaotic light, the correlation function of Equation (27) contains much more information than the ghost image: the main physical differences are once again related with the coherent nature of the images it encodes.

Coherence is the immediate consequence of measuring coincidences between the spatial sensor ${\mathrm{D}}_a$ and any pixel of the angular sensor ${\mathrm{D}}_b$. This can be better understood in terms of the Klyshko picture [75] reported in Figure 13: light illuminating the object and contributing to the coincidence detection between any two pairs of pixels ${\mathit{\rho }}_a$ and ${\mathit{\rho }}_b$ has a well defined propagation direction; hence, it is responsible for the formation of a coherent image.

Now, to explicitly demonstrate both the refocusing and three-dimensional imaging capabilities of this scheme, and to better highlight the plenoptic properties of the correlation function of Equation (27), we shall consider the more general out-of-focus situation ($z_b\ne z_{bF}$). The stationary points of the phase defined in Equation (28) enable to determine the geometrical correspondence between points on the object and the source with points on sensors ${\mathrm{D}}_a$ and ${\mathrm{D}}_b$, respectively. In particular, the stationarity of $\phi$ with respect to ${\mathit{\rho }}_s$ determines the object point that gives the predominant contribution to the integral of Equation (27), which is

$${\mathit{\rho }}_o=-\frac{z_b}{z_{bF}}\frac{{\mathit{\rho }}_a}{m}-\frac{{\mathit{\rho }}_b}{M}\left(1-\frac{z_b}{z_{bF}}\right).$$

(31)

When the focusing condition of Equation (29) is satisfied, this object point becomes independent of the specific sensor pixel ${\mathit{\rho }}_b$; hence, the focused ghost image is not sensitive anymore to the change of perspective enabled by the high resolution of the angular sensor ${\mathrm{D}}_b$. On the other hand, the stationarity of $\phi$ with respect to ${\mathit{\rho }}_o$ yields the focusing of the source on the sensor ${\mathrm{D}}_b$:

$${\mathit{\rho }}_s=-\frac{{\mathit{\rho }}_b}{M}.$$

(32)

Therefore, in the geometrical optics limit, the correlation function of Equation (27) reduces to the product of the tilted and rescaled geometrical image of the object with the source profile

$${\mathsf{\Gamma }}_{(z_a^{\prime },z_b)}({\mathit{\rho }}_a,{\mathit{\rho }}_b)\sim {\left|A\left[-\frac{z_b}{z_{bF}}\frac{{\mathit{\rho }}_a}{m}-\frac{{\mathit{\rho }}_b}{M}\left(1-\frac{z_b}{z_{bF}}\right)\right]\right|}^2{\left|\mathcal{F}\left(-\frac{{\mathit{\rho }}_b}{M}\right)\right|}^2.$$

(33)

By properly rescaling the variable ${\mathit{\rho }}_a$, the correlation function gives the perfectly aligned geometrical images of the desired scene, one for each value of ${\mathit{\rho }}_b$, namely

$${\mathsf{\Gamma }}_{(z_a^{\prime },z_b)}^{\mathrm{ref}}\left[\frac{z_{bF}}{z_b}{\mathit{\rho }}_a+\frac{{\mathit{\rho }}_b}{M}m\left(1-\frac{z_{bF}}{z_b}\right),{\mathit{\rho }}_b\right]\sim {\left|\mathcal{F}\left(-\frac{{\mathit{\rho }}_b}{M}\right)\right|}^2{\left|A\left(-\frac{{\mathit{\rho }}_a}{m}\right)\right|}^2.$$

(34)

Such rescaling is formally identical to the one employed both in standard plenoptic imaging [3] and in correlation plenoptic imaging with chaotic light (see Equation (11)). Similar to standard plenoptic imaging, the signal-to-noise ratio is highly improved by integrating the result of Equation (34) over the whole sensor array ${\mathit{\rho }}_b$, thus employing light coming from the whole light source. The result

$${\mathsf{\Sigma }}_{(z_a^{\prime },z_b)}^{\mathrm{ref}}({\mathit{\rho }}_a)=\int d{\mathit{\rho }}_b{\mathsf{\Gamma }}_{(z_a^{\prime },z_b)}^{\mathrm{ref}}\left[\frac{z_{bF}}{z_b}{\mathit{\rho }}_a+\frac{{\mathit{\rho }}_b}{M}m\left(1-\frac{z_{bF}}{z_b}\right),{\mathit{\rho }}_b\right]$$

(35)

represents the refocused incoherent ghost image of an object placed at a generic distance $z_b$ from the source.

The accuracy of the quasi one-to-one correspondence of object and source points with their images on ${\mathrm{D}}_a$ and ${\mathrm{D}}_b$, respectively, is at the basis of the resolution properties of CPI with entangled photons. On one hand, the object PSF in Equation (30) is characterized by a spot size $\Delta {\rho }_a\sim mc{z}_{bF}/(\mathsf{\Omega }{D}_s)$, with $D_s$ the typical transverse size (effective diameter) of the pump profile. On the other hand, the PSF of the source image is determined by the size a of the smallest object details, through $\Delta {\rho }_b\sim Mc{z}_b/(\mathsf{\Omega }a)$. Hence, for pixel size above the resolution limits, spatial and directional resolutions are decoupled, as in chaotic light CPI. Therefore, also CPI with entangled photons overcomes the intrinsic limitations of standard plenoptic imaging and achieves a larger depth of field (as defining by the angular resolution), with diffraction-limited resolution.

5.2. Depth-of-Field Improvement

Let us compare the depth of field of CPI with entangled photons with the one of PI and SI devices, following the same strategy as in Section 2.3, namely by evaluating the visibility of the images of a double-slit mask having width a and center-to-center distance $d=2a$, as retrieved at different axial positions $z_b$, The minimum resolved distance d is defined as the one at which the visibility of the slits image falls below 10%.

In Figure 14a, we plot the minimum resolved distance d, as a function of the defocusing parameter $z_b-z_{bF}$, for ghost imaging (orange), plenoptic imaging with $N_u=3$ (green) and CPI (blue). In all cases, we choose the same numerical aperture ($\mathrm{NA}=0.3$). CPI clearly enables refocusing in a wider range than both standard imaging (SI) and standard plenoptic imaging (PI) without losing the high resolution of the focused image. Points A and B correspond to the examples shown in Figure 14b. In case A, the slit distance and $z_b$ are chosen to be very close to the boundary of the CPI refocusing range: both SI and PI are completely out-of-focus, while CPI still partially resolves the object. Point B lies outside the PI refocusing range but well below the CPI limits, and the refocused image thus appears to be well-resolved. Interference effects in the simulated results of Figure 14b are due to the coherent nature of CPI.

6. Conclusions

We have reviewed the basic principles and most relevant implementations of correlation plenoptic imaging, while focusing both on the improvement with respect to standard plenoptic imaging and the comparison among different CPI strategies. Unlike standard plenoptic imaging, CPI has no constraints on image resolution, which stays diffraction limited as in standard imaging systems. At the same time, CPI enables increasing the DOF well beyond the typical values of standard imaging, as reported in Figure 7 and Figure 14. The results are unchanged for reflective and transmissive samples. CPI is also expected to work with different kinds of sources, of either photons or particles [77], provided they are characterized by correlation in both position and momentum [61,78].

CPI is promising for both microscopy and 3D imaging, due to its ability to decouple transverse and axial resolutions, to achieve high DOF, and to acquire multiperspective images with a single device. In the future, we will employ both hardware (fast CMOS, smart sensors [79]) and software (compressed-sensing and sparse measurement techniques [80]) solutions to optimize the acquisition time and regain the single-shot advantage of conventional plenoptic imaging. Moreover, we will investigate the potential of the intrinsically coherent nature of CPI, to achieve superresolution and perform tasks, like wavefront sensing, in which quantum state estimation techniques [81,82], based on the reconstruction of the coherence function of light, have already proved to be effective.

7. Patents

• Device and process for the plenoptic capture of images, request n. 102016000027106 of 15 March 2016 to the Italian Patent Office (approved); extension request n. EP17160543.9 of 13 March 2017 to the European Patent Office (pending); inventors: Milena D’Angelo, Augusto Garuccio, Francesco V. Pepe, Teresa Macchia, Ornella Vaccarelli.
• Device and process for the contemporary capture of standard and plenoptic images, request n. PCT/IB2017/055842 of 26 September 2017 to the International Searching Authority (pending); inventors: Milena D’Angelo, Augusto Garuccio, Francesco V. Pepe, Ornella Vaccarelli.
• Device and process for the acquisition of microscopic plenoptic images with turbulence mitigation, request n. 102018000007857 of 3 August 2018 to the Italian Patent Office (pending); inventors: Milena D’Angelo, Francesco Di Lena, Augusto Garuccio, Francesco V. Pepe, Alessio Scagliola.