Research on Multipath Correlated Imaging with the Grayscale Target in Endoscopic Applications

Abstract

The application of correlated imaging in endoscope, one of the research hotspots, may lead to multipath effect in the closed endoscopic environment. The model of multipath correlated imaging with a grayscale object is given, where the mismatch ratio and the reflection ratio are two key factors affecting imaging quality. The theoretical and experimental results show that multipath effect has an influence on the grayscale distribution and imaging quality of the reconstructed image, and the effect of the mismatch ratio is stronger than that of the reflection ratio. The conditions that the disturbance from multipath effect can be ignored in endoscopic applications are given.

1. Introduction

Correlated imaging (CI) has been studied extensively in the past few decades, both experimentally and theoretically [1,2,3,4,5,6,7,8]. It was first realized by using the entangled photon pairs created by spontaneous parametric down-conversion (SPDC) in 1995 [1]. Bennink et al. confirmed that entanglement is not a prerequisite for CI, and provided an experimental demonstration with a classical source [6]. Cheng and Han studied a particular aspect of coincidence imaging with incoherent source by using classical statistical optics [7]. After that, theoretical and experimental CI schemes exploiting the correlation of two classical beams obtained by splitting incoherent thermal source were proposed [9,10,11,12,13,14,15,16,17]. Furthermore, many relevant results including high-order [18,19], computational [20,21], differential [22], and compressive sensing [23,24] were also reported. CI have been achieved and CI has found its applications ranging from super-resolution [25] to remote sensing [26], to medical imaging [27,28].

In recent years, many scholars at home and abroad have studied the key metrics of correlated imaging, such as resolution and robustness. The results show that correlated imaging not only has the advantage of super-resolution [29,30], but also has strong robustness [31,32]. Only part of the object light is needed to reconstruct the object image. These advantages pave the way for the application of endoscopes. While in the endoscopic environment, the self-interference caused by multipath effect has a great influence on imaging quality. Thus, in this paper, we first establish the model of correlated imaging with multipath effect, and the grayscale object is used to simulate tissue in different physiological states. Then, we give the analytical expression of the point spread function (PSF) of CI when the multipath effect is considered, from which one can see that the influence of the mismatch ratio and reflection ratio on imaging quality are quite different. Based on these two parameters, we theoretically and experimentally investigate the influence of multipath effect on the grayscale distribution of the reconstructed image.

2. Model and Theoretical Analysis of Multipath Correlated Imaging

Multipath can be divided into primary reflection multipath and higher-order multipath. Among them, primary reflection multipath has the strongest energy in multipath signals [33,34,35], so primary reflection multipath is considered in the following analysis. Figure 1a shows a multipath model of endoscope application in the closed environment. The signals received by the receiving port consist of one direct signal and several reflected signals [36]. Due to the reflection, the path length of the multipath signal is longer than that of the direct signal. What is more, the energy of multipath signals is usually much weaker than that of the direct signal on account of reflection. For simplicity, we treat all multipath signals as one path and combine it with CI to obtain the model of multipath CI, as shown in Figure 1b. It should be emphasized that when the time delay is larger than the coherence time of the source, there is inter-pulse disturbance. On the contrary, there is intra-pulse disturbance. In this case, the disturbance of multipath effect on the image is the most serious. So, intra-pulse disturbance is discussed in this paper. The classical thermal light source, usually obtained by illuminating a continuous laser into a slowly rotating ground glass, is divided into three beams by a combination of two beam splitters (BS). The three beams travel through a reference arm and two test arms, which are characterized by their impulse response function h₂(x₂,u₂), h₁(x₁,u₁), h₃(x₁,u₁), respectively. The test arm includes a grayscale object to be imaged, which is located at a distance z₂ from the bucket detector (D₁). As for the reference arm, the light propagates freely z length to a reference detector (D₂) with spatial resolution. Detectors D₁ and D₂ are used to record the intensity distribution at u₁ and u₂, respectively. In the test arm, the light may directly reach the object (see the dashed line z₁) or it may be reflected by the reflector and then reach the object (see the dashed line z₃). The reflected light interferes with the direct light during the imaging process, which may affect imaging quality. To get a deeper insight into the influence of multipath effect on CI, two parameters are defined, namely mismatch ratio (η) and reflection ratio (ρ). The former is defined as (z₃ − z₁)/z₁, and the latter is defined as the ratio of power between the reflected light and the total object light.

Using a correlator, all the information about the object can be constructed by the correlation function between the intensity fluctuations,

$$\begin{array}{cc}G(u_1,u_2)& ={\left|{\displaystyle \int \Gamma (x_1,x_2)h_1(x_1,u_1)h_2^{*}(x_2,u_2)d{x}_{1}d{x}_2}\right|}^2\\ & +{\left|{\displaystyle \int \Gamma (x_1,x_2)h_3(x_1,u_1)h_2^{*}(x_2,u_2)d{x}_{1}d{x}_2}\right|}^2\end{array}$$

(1)

where x_i (i = 1, 2) is the location of the source plane, Γ_i (x₁,x₂) (i = 1, 2) represents the first-order correlation function of the source.

Suppose that the light source is fully spatially incoherent, then the first-order correlation function of the source can be written as:

$$\Gamma (x_1,x_2)=\exp \left(-\frac{x_1^2+x_2^2}{2a^2}\right)\delta (x_1-x_2) ,$$

(2)

where a is the transverse size of the source and $\delta \left(\cdot \cdot \cdot \right)$ is the Dirac delta function.

In the reference arm, D₂ is located at a distance z from the source. Therefore, the impulse response function can be given by the Fresnel integral,

$$h_2(x_2,u_2)={\left(\frac{-i}{\lambda z}\right)}^{-1/2}\exp \left[\frac{-i\pi }{\lambda z}{(u_2-x_2)}^2\right].$$

(3)

As for the test arm, there are two propagation paths z₁ and z₃, and the corresponding impulse response functions are

$$h_1(x_1,u_1)={\left(\frac{-1}{{\lambda }^2{z}_1{z}_2}\right)}^{-1/2}{\displaystyle \int \exp \left[\frac{-i\pi }{\lambda z_1}{(y-x_1)}^2\right]}t(y)\times \exp \left[\frac{-i\pi }{\lambda z_2}{(u_1-y)}^2\right]dy,$$

(4)

$$h_3(x_1-x_0,u_1)={\left(\frac{-1}{{\lambda }^2{z}_2{z}_3}\right)}^{-1/2}{\displaystyle \int \exp \left(\frac{-i\pi }{\lambda z_3}{\left[y-\left(x_1-x_0\right)\right]}^2\right)}t(y)\times \exp \left[\frac{-i\pi }{\lambda z_2}{(u_1-y)}^2\right]dy,$$

(5)

where the coordinate x₀ represents the deviation of the pseudo-image relative to the real image at both horizontal and vertical distances. Substituting Equations (2)–(5) into Equation (1), the simplified correlation function of intensity fluctuation G(u₂) can be obtained as

$$\begin{array}{ll}G(u_2)& =\frac{{\beta }_0{\beta }_2}{{\pi }^3}{\int dy\left|t\left(y\right)\right|}^2\left[{\beta }_1{h}^{\prime }(y,u_2)+{\beta }_3{h}^{″}(y,u_2)\right]\\ & =G_1(u_2)+G_2(u_2)\end{array},$$

(6)

where $\alpha =\frac{1}{a^2},{\beta }_0=\frac{\pi }{\lambda z},{\beta }_1=\frac{\pi }{\lambda z_1},{\beta }_2=\frac{\pi }{\lambda z_2},$ and ${\beta }_3=\frac{\pi }{\lambda z_3}$. From Equation (6), one can see that the reconstructed image consists of two terms. The first term represents the real image which is from the correlation between the direct light and the reference light, and the second term corresponds to the defocused image formed by correlating the reflected light with the reference light. From Equation (6), the PSF of the multipath CI system can be written as:

$$\begin{array}{ll}h\left(y\right)& =h^{\prime }(y)+\rho h^{″}(y)\\ & =\frac{\pi }{\frac{1}{a^2}}\exp \left[\frac{-y^2}{\frac{{\lambda }^2{z}_1^2}{2a^2{\pi }^2}}\right]+\rho \frac{\pi }{\frac{1}{a^2}\sqrt{1+\frac{{\pi }^2{a}^4}{{\lambda }^2}{\left(\frac{1}{z}-\frac{1}{z_3}\right)}^2}}\\ & \times \exp \left[\frac{-y^2-2x_{0}y-x_0^2}{\frac{{\lambda }^2{z}_3^2}{2a^2{\pi }^2}\left[1+\frac{{\pi }^2{a}^4}{{\lambda }^2}{\left(\frac{1}{z}-\frac{1}{z_3}\right)}^2\right]}\right]\end{array}.$$

(7)

According to Equation (7), it clearly emerges that there are mainly three parameters that affect imaging quality of the reconstructed image: the path length z₃ of reflected light, reflection ratio ρ, and the deviation x₀. It should be emphasized that when there exists deviation, the defocused image and the real image will overlap partly or not overlap. While the two images will overlap completely when there is no deviation. In this case, the defocused image has the greatest interference on the reconstructed image. Thus, in the following, we will mainly discuss the influence of η and ρ on the reconstructed image. Based on Equation (7), the normalized PSF is depicted in Figure 2. It is well known that the resolution is dependent on the width of PSF. The smaller the width is, the higher the resolution is. From Figure 2a, it is shown that the width of PSF is the smallest when there is no multipath effect, and the profile becomes broader with the increase of η. From Figure 2b, it is found that the increase of the power of the reflected light results in a decrease of imaging resolution. It is also shown that the reflection ratio has smaller effect than the mismatch ratio by comparing Figure 2a,b.

3. Numerical and Experimental Results

In this section, we use the numerical simulations and experiments to further analyze the influence of multipath effect on CI with a grayscale target in endoscopic applications. The experimental setup is shown in Figure 1b. The light source, made by a 532 nm laser passing through rotating ground glass, is split into three parts by a combination of two 50%–50% nonpolarizing beam splitters. The transverse size of the source a = 1 mm. In the object arm, one way of light passes through the object directly, and the other way of light passes through the object after being reflected by the reflector. The object light is detected by a bucket detector (Thorlabs PDA100A2). In the reference arm, the light freely propagates to the charge-coupled device (CCD) detector (Thorlabs DDC3240C), exposure time of the CCD is 0.2 ms. As shown in Figure 3a, we choose a grayscale image with four slits whose grayscale changes from light to dark and the levels are clearly divided as the target object. It is not only convenient to investigate the grayscale reconstruction ability of multipath correlated imaging, but also convenient to calculate the resolution. The slit width w = 0.3 mm and the distance between two adjacent slits d = 0.3 mm. The resolution can be defined as:

R_i = Y_i/X_i (i = 1, 2, 3).

(8)

where Y_i represents the maximum grayscale value of the slit, X_i represents the maximum grayscale value of the adjacent slit spacing. According to the model of the multipath CI system, the object light is divided into two paths, and the path length of reflected light is longer than that of the direct light. In the following, we will first discuss the effect from the path length of the reflected light, i.e., the mismatch ratio, on the grayscale values of the reconstructed image, and the corresponding results are given in Figure 3b–g. The solid curves in the lower are the normalized horizontal sections of the patterns in the upper and the dashed curves correspond to the original object. It is obvious that the grayscale values of the reconstructed image vary with the mismatch ratio, which is mainly due to the existence of the defocused image.

To explain the phenomenon in Figure 3, the defocused images under different mismatch ratios are depicted in Figure 4. The upper are the numerical simulation results obtained over 5000 CCD frames, and the lower are the corresponding normalized horizontal sections. For the sake of comparison, we give the numerical simulation about the real image in Figure 4a, i.e., the image when there is no multipath effect. It should be noticed that by comparing the curves in Figure 4a–f, the defocused image gradually becomes smaller and the part with higher grayscale value moves towards the left with an increase of η.

By comparing Figure 4a with Figure 4b–f, it is found that as η increases, the maximum grayscale value of the defocused image gradually moves from the fourth slit of the real image to the third slit, so the normalized intensity of the fourth slit in the reconstructed image decreases from 1.0 to 0.67 approximately, as shown in Figure 3b,d,g, and the normalized intensity of the third slit increases from 0.78 (see Figure 3b) to 1.0 (shown in Figure 3d), and then remains unchanged.

To illustrate the change of grayscale distribution more clearly, we give the dependence of the normalized intensity for each slit on η in Figure 3h. When η < 0.3, the grayscale values of the second and the third slits increase. After η reaches 0.3, the grayscale value of the third slit exceeds that of the fourth slit, so the image is significantly distorted. When η < 0.3, the grayscale distribution has slightly fluctuation, thus the disturbance of multipath effect can be ignored. It clearly emerges that when the mismatch ratio increases, a defocused image with the decreasing size is generated, which results in a decrease of the superposition area between the defocused image and the real image. At this point, the conclusion is drawn that the variation area of grayscale values for the reconstructed image is mainly determined by the mismatch ratio. It should be emphasized that, the deviation of the defocused image relative to the real image is 0 in Figure 3. For the case with the existence of the deviation, the maximum grayscale value may appear in other locations. To verify the correctness of the PSF in Figure 2a, the dependence of η on imaging resolution is given in Figure 3i. It is shown that the resolution presents a downward trend with an increase of η.

Then, we focus on another parameter, the reflection ratio, which determines the intensity change of the reflected light. Shown in Figure 5a–d are the reconstructed images for different reflection ratios when η is 0.8. Because of energy attenuation in the process of the reflection, the power of reflected light is generally much lower than that of direct light, so the range of reflection ratio is chosen as $0\le \rho \le 0.5$. The increase of ρ aggravates the interference caused by the defocused image, so the edges of the reconstructed image in Figure 5a–d appear somewhat blurred.

The dependence of the normalized intensity of four slits on ρ is given in Figure 5e. When ρ < 0.2, the fluctuation of grayscale distribution is quite small. The grayscale value of the third slit gradually reaches the maximum. As shown in Figure 5a–e, the increase of ρ lead to significant distortion of the image, and it is obvious that the variation amplitude of grayscale values for the reconstructed image increases with the increase of the reflection ratio. From the above discussion, the disturbance caused by multipath effect can be ignored when ρ < 0.2. By comparing Figure 3 with Figure 5, we can draw the conclusion that the influence of η on the grayscale distribution is bigger than that of ρ. The influence of ρ on imaging resolution is also analyzed, and the results are plotted in Figure 5f. Obviously, imaging resolution dramatically decreases with the increasing ρ, while the degree of decline is smaller than that in Figure 3i, which indicates that the effect from η is bigger than that from ρ.

For the sake of verifying the changes on the grayscale distribution and the resolution in numerical simulation, we implement CI with multipath effect experimentally. The corresponding results obtained over 5000 CCD frames are shown in Figure 6. The experimental results are in agreement with the above theoretical results, i.e., the conclusion about the variation of grayscale distribution of the reconstructed image is in accordance with the theoretical results, and η has a stronger influence on CI with multipath effect when compared with ρ.

4. Conclusions

In conclusion, we have theoretically and experimentally investigated the influence of multipath effect on CI with a grayscale object in endoscopic applications. When the multipath effect exists, the path length of the reflected light is larger than that of the direct light. What is more, the power of reflected light is much weaker than that of the direct light due to the reflection. Based on this theory, the model of CI with multipath effect is established. The mismatch ratio and the reflection ratio are defined. It is noted that the time delay is smaller than the coherent time of the source in this paper. In this case, the disturbance of multipath effect on the reconstructed image is the most serious. For the case that the time delay is larger than the coherent time of the source, the background noise may be enhanced by the reflected light, which may affect the imaging of object with lower grayscale values. This case will be discussed in the following work.

The results show that a multipath effect has an influence on the grayscale distribution and imaging quality, and the effect from the mismatch ratio is stronger than that from the reflection ratio. As the mismatch ratio increases, a defocused image with the decreasing size and poor resolution is generated, meaning the decreased variation area of the grayscale values for the reconstructed image. For the larger reflection ratio, the variation amplitude of grayscale values for the reconstructed image increases due to the enhanced disturbance of the defocused image. It is found that the disturbance from multipath effect can be ignored when the mismatch ratio is smaller than 0.3 or the reflection ratio is smaller than 0.2. Thus, in the closed endoscopic environment, the disturbance of multipath effect on the reconstructed image can be ignored within a certain range. Our study paves the way for practical endoscopic applications.