Advanced Visualization Polarimetric Imaging: Removal of Water Spray Effect Utilizing Circular Polarization

Abstract

Circular polarization (CP) memory is a well-known phenomenon whereby natural light becomes partially circularly polarized after scattering by water spray several times, and the circularly polarized state can be well preserved within a certain propagation distance. In this study, a CP imaging method combined with the multi-scale analysis in the frequency domain is proposed to enhance the vision in rainy conditions. The images were first decomposed into multi-scales. CP characteristics of light were employed in the low-frequency parts to improve the quality of images in rainy conditions, and the high-frequency parts compensated specific structure information. Experimental results demonstrate that the proposed method can remove the water spray effect thereby improving the vision of degraded rainy-day images.

1. Introduction

Techniques for imaging through turbid media have recently received considerable attention because of their potential applications such as atmosphere remote sensing, underwater photography, and even biomedical imaging [1,2,3]. The major difficulty encountered in these techniques is the loss of directionality of the incident light caused by multiple scattering among particles (water spray, dust, etc.), which leads to degradation of the image quality [4,5,6].

Several studies that focused on polarimetric imaging techniques are of great interest [7,8,9]. By retaining and displaying polarization differences between airlight (natural illumination) and the direct transmission of light through an object in polarization images, polarimetric imaging can make it possible to enhance the visibility and contrast of images taken through turbid media. Schechner, Fang, Liu, and collaborators [10,11,12,13] mainly focused on image dehazing by employing linearly polarized information on light based on the fact that airlight is partially polarized light that can be detected. However, linear polarization (LP)-based methods rarely show satisfactory recovery results when it comes to rainy-day images. Two major reasons account for this phenomenon: one is the attenuation of thick clouds on rainy days, which causes a sharp reduction in the amount of light captured by cameras, but the circular polarization (CP) information in the scene remains stable based on the CP memory effect; the other is the characteristic of water spray, by which the interaction between airlight and water spray can turn natural light into circularly polarized light [14,15,16]. Moreover, since CP light is point-symmetric, it does not depend on the scattering angle, and can survive more multi-scattering events than linearly polarized light does [17]. The CP light maintains better polarization characteristics than LP for both forward and backward scattering, as shown in Figure 1a. The effects of size on the degree of polarization are shown in Figure 1b. The degree of polarization (DOP) is greater for LP than for CP for small sizes. When the size parameter is large, such as with a water drop, the DOP for CP is greater than for LP [18]. The circular polarization characteristics of light have been used in an effort to improve the image quality of underwater targets [19]. It provides another possible way to compensate for the deficiency of LP-based technologies in enhancing rainy-day image quality.

In this study, we first analyzed limitations of traditional polarization image recovery methods based on linearly polarized light and then reported a CP imaging method combining the multi-scale analysis to image in rainy conditions and enhance visibility. Several scenes on a rainy day were captured using CP imaging devices. Then, the images were decomposed into various spatial frequency layers using wavelet transform. This operation concentrates the polarization information from objects and water spray to the low-spatial frequency components of the image; meanwhile, the specific structure information is concentrated to the high-spatial frequency components of the image. Furthermore, the degraded image could be recovered by employing the CP characteristics from the low-spatial frequency components. At the same time, the high-spatial frequency components were manipulated to reconstruct the details of the images by the nonlinear transform method. Experiments in rainy-day conditions were performed to demonstrate the effectiveness of the proposed method.

2. Materials and Methods

2.1. Optical Model of Rainy Conditions

As shown in Figure 2, the thick clouds heavily reduce the intensity of natural light, which leads to a reduction in image brightness. In addition, along with the rain, there is an abundance of water spray in the air that generates scattering of light [16]. The reflected light of different bands has different transmissivity mapping. Hence, images taken through this kind of scattering media are characterized by a loss of contrast and changes in color. Even though there is serious image degradation, the specific atmospheric conditions on rainy days provide the possibility to image through the rain by employing circularly polarized information. Several studies on the propagation of light through water spray have demonstrated that for randomly oriented particles [15,16,17], the incident light (sunlight before into the water spray) [S₀(x, y), S₁(x, y), S₂(x, y), S₃(x, y)]^T and the scattered light (reflected light from target surface) [S₀⁰(x, y), S₁⁰(x, y), S₂⁰(x, y), S₃⁰(x, y)]^T are related by Equation (1):

$$\begin{array}{l}{\left[S_0(x,y),S_1(x,y),S_2(x,y),S_3(x,y)\right]}^T=L\left(\pi -i_2\right)\cdot P\left(\psi \right)\cdot L\left(-i_1\right)\cdot {\left[S_0^0(x,y),S_1^0(x,y),S_2^0(x,y),S_3^0(x,y)\right]}^T\\ =\left(\begin{array}{cccc}{p}_{11}(\psi )& p_{21}(\psi )\cos 2i_1& -p_{21}(\psi )\sin 2i_1& 0\\ p_{21}(\psi )\cos 2i_2& \begin{array}{l}{p}_{22}(\psi )\cos 2i_1\cos 2i_2\\ -p_{33}(\psi )\sin 2i_2\sin 2i_1\end{array}& \begin{array}{l}-(p_{22}(\psi )\cos 2i_2\sin 2i\\ +p_{33}(\psi )\sin 2i_2\cos 2i_1)\end{array}& p_{43}(\psi )\sin 2i_2\\ p_{21}(\psi )\sin 2i_2& \begin{array}{l}{p}_{22}(\psi )\sin 2i_2\cos 2i_1\\ +p_{33}(\psi )\times \sin 2i_1\cos 2i_2\end{array}& \begin{array}{l}{p}_{33}(\psi )\cos 2i_2\cos 2i_1\\ -p_{22}(\psi )\sin 2i_1\sin 2i_2\end{array}& -p_{43}(\psi )\cos 2i_2\\ 0& p_{43}(\psi )\sin 2i_1& p_{43}(\psi )\cos 2i_1& p_{44}(\psi )\end{array}\right)\left(\begin{array}{c}{S}_0^0(x,y)\\ S_1^0(x,y)\\ S_2^0(x,y)\\ S_3^0(x,y)\end{array}\right),\end{array}$$

(1)

where p is the phase matrix; Ψ means the scattering angle; i₁ and i₂ are the angles of rotation of the light scattered by water molecules; L is the rotation matrix; (((S₁(x, y))² + (S₂(x, y))²)^1/2/S₀(x, y)) is the Degree of LP (DoLP); (S₃(x, y)/S₀(x, y)) is the Degree of CP (DoCP) [15]. The LP light ([S₀¹(x, y), S₁¹(x, y), 0, 0]^T) exists after a single scattering with sunlight as the incident light. It is also possible for CP light to be generated when the incident light experiences multiple scattering according to Equation (1). Können [16] also stated the fact that a rainy day or watery environment is a very specific condition that can generate more circularly polarized light through multi-scattering in a natural environment compared to a sunny day. Under natural conditions, multiple scattering is much more common than single scattering in media [15,18]. Taking two-particle scattering as an example (Figure 3), the CP component S₃²(ψ, ρ) can be written as Equation (2):

$$S_3^2\left(\psi ,\rho \right)=\frac{2P_{21}\left(\psi \right)P_{43}\left(\psi \right)\sin \psi \cdot \sin \rho \cdot \cos \rho }{\left(1-{\cos }^2\psi \cdot {\cos }^2\rho \right)}.$$

(2)

where ρ is the angle of the incident light plane and reflected light plane. According to typical single scattering by a droplet and Brewster’s Law, the emergent light can reach an impressively high DoLP (more than 90%) when light is incident at an ideal position (generally, the incident angle stays between 38° and 43°, and the emergent angle ranges from 48° to 60°) [16]. The light with high DoLP is more likely to generate CP light after scattering [15]. Furthermore, in media consisting of particles comparable in size to the wavelength of natural light, the CP light can survive more multi-scattering events than linearly polarized light [20,21,22,23], which is called the polarization memory effect. Since circularly polarized light is point-symmetric, it does not depend on the scattering angle. For example, right circularly polarized light that undergoes scattering of any angle maintains its exact Stokes vector, while the partially linearly polarized light that undergoes the same scattering angle will have its polarization state changed necessarily. The analyses above make it a sound foundation to employ the CP information of scattered light to images in rainy conditions.

2.2. Water Spray Removal

According to the CP characteristics, the CP light can be detected by setting up a measurement system as shown in Figure 4. Two key parts of the measurement system include a retarder and a linear polarizer, as shown in Figure 4. The imaging process of the system can be expressed by the Mueller matrix M_R(ϕ, θ_R, θ_P) in Equation (3):

$$\begin{array}{l}M=M_R(\varphi ,{\theta }_R)\cdot M_P({\theta }_P)\\ =\frac{1}{2}\left[\begin{array}{cccc}1& 0& 0& 0\\ 0& \begin{array}{l}{\cos }^{2}2{\theta }_R+\\ \cos \varphi {\sin }^{2}2{\theta }_R\end{array}& \begin{array}{l}(1-\cos \varphi )\times \\ \sin 2{\theta }_R\cos 2{\theta }_R\end{array}& -\sin \varphi \sin 2{\theta }_R\\ 0& \begin{array}{l}(1-\cos \varphi )\times \\ \sin 2{\theta }_R\cos 2{\theta }_R\end{array}& \begin{array}{l}{\sin }^{2}2{\theta }_R+\\ \cos \varphi {\cos }^{2}2{\theta }_R\end{array}& \sin \varphi \cos 2{\theta }_R\\ 0& \sin \varphi \sin 2{\theta }_R& -\sin \varphi \cos 2{\theta }_R& \cos \varphi \end{array}\right]\cdot \left[\begin{array}{cccc}1& \cos 2{\theta }_p& \sin 2{\theta }_p& 0\\ \cos 2{\theta }_p& {\cos }^{2}2{\theta }_p& \begin{array}{l}\sin 2{\theta }_p\times \\ \cos 2{\theta }_p\end{array}& 0\\ \sin 2{\theta }_p& \begin{array}{l}\sin 2{\theta }_p\times \\ \cos 2{\theta }_p\end{array}& {\sin }^{2}2{\theta }_p& 0\\ 0& 0& 0& 0\end{array}\right],\end{array}$$

(3)

where M_R(ϕ, θ_R) and M_P(θ_P) represent the Mueller matrix of the retarder and the linear polarizer at an arbitrary angle, respectively; ϕ is the total phase shift between the two orthogonal components of the incident field; θ_R denotes the angle between the fast axis of the retarder and x-axis, and; the transmission axis of the polarizer is set at angle θ_P to the x-axis. During measurement, ϕ is fixed to 90° and θ_R is set to 45° in order to derive Equation (4). Then, the transformation of CP to LP, its detection by rotating the linear polarizer to change θ_P, and two states (I_B(x, y, θ_P) and I_W(x, y, θ’_P)) of the incident light—named the best state and the worst state—are determined [8]. When calculating the airlight I_A^∞(x, y), we choose the intensities of areas without any objects in the best state and worst state images and mark them by ${\widehat{I}}_B\left(x,y,0^{\circ }\right)$ and ${\widehat{I}}_W\left(x,y,{90}^{\circ }\right)$, respectively. Equation (5) gives the DoCP of the airlight, according to the definition of polarization degree.

$$I(x,y,{\theta }_p)=\frac{1}{2}\left[S_0(x,y)+S_3(x,y)\cdot \cos \left(2{\theta }_p\right)\right],$$

(4)

$$p_c=\left|\frac{{\widehat{I}}_B(x,y,0^{\circ })-{\widehat{I}}_W(x,y,{90}^{\circ })}{{\widehat{I}}_B(x,y,0^{\circ })+{\widehat{I}}_W(x,y,{90}^{\circ })}\right|=\left|\frac{S_3(x,y)}{S_0(x,y)}\right|.$$

(5)

As we know, a wavelet transform decomposes an image into low- and high-frequency components, where a low-frequency component denotes the approximation, and a high-frequency component characterizes the details of its last order image. Accordingly, the circular polarization information brought in by the interaction between raindrops and natural light is taken care of in the low-frequency parts [17]. In the captured images, the radiance I_T(x, y, λ) received by the detector can be expressed as Equation (6) in the low-spatial frequency parts.

$$I_T\left(x,y,\lambda \right)=I_D\left(x,y,\lambda \right)+I_A\left(x,y,\lambda \right),$$

(6)

where I_D(x, y, λ) represents the direct transmission of radiance from the imaged scene; I_A(x, y, λ) denotes the radiance of the airlight; I_O(x, y, λ) stands for the object radiance in a clear-day image, as shown in Equation (7) [10]. Additionally, t(x, y, λ) = e^{−βz(x,y,λ)} means the atmosphere-attenuated coefficient, where β is the scattering coefficient [4]; I_A^∞(x, y, λ) is the airlight radiance corresponding to an object at infinity. λ is the wavelength.

$$I_O\left(x,y,\lambda \right)=\frac{I_T\left(x,y,\lambda \right)-I_A\left(x,y,\lambda \right)}{t\left(x,y,\lambda \right)}=\frac{I_T\left(x,y,\lambda \right)-I_A\left(x,y,\lambda \right)}{1-\left(I_A\left(x,y,\lambda \right)/I_A^{\infty }\left(x,y,\lambda \right)\right)}.$$

(7)

Two key parameters, I_A(x, y, λ) and I_A^∞(x, y, λ), should be considered to find I_O(x, y, λ). The I_A(x, y, λ) can be expressed as I_A(x, y, λ) = (S₀(x, y, λ)·p_c)/p_c^∞, where p_c^∞ is the DoCP corresponding to an object at infinity. Therefore, the core of this open problem is to calculate the intensity of airlight by utilizing the CP characteristics of scattered light. In combination with Equations (5) and (7), the final way to calculate the object radiance in a clear-day image I_O(x, y, λ) is expressed by Equation (8).

$$I_O\left(x,y,\lambda \right)=\frac{p_c\cdot I_T\left(x,y,\lambda \right)-\Delta \widehat{I}(x,y,\lambda )\cdot \left(1-t\left(x,y,\lambda \right)\right)}{p_c\cdot t\left(x,y,\lambda \right)},$$

(8)

where $\Delta \widehat{I}(x,y,\lambda )$ = (I_B(x,y,θ_P) − I_W(x,y,θ’_P)). Although the wavelet decomposition enables the image recovery in the low-frequency components, the absolute values of the wavelet coefficients become larger, which would result in a decrease in image clarity in high-spatial frequency regions [10]. Hence, a dual-threshold process algorithm is taken to adjust the high-frequency coefficients, as shown in Equation (9). This can remove the coefficients within a certain threshold range and enhance those outside the threshold range.

$$W_{out}=\{\begin{array}{ll}{W}_{in}+\left(T_2\cdot \left(G-1\right)\right)-\left(T_1-G\right)& W_{in}>T_2\\ G\cdot \left(W_{in}-T_1\right)& T_1<W_{in}<T_2\\ 0& -T_1\le W_{in}<T_1\\ G\cdot \left(W_{in}+T_1\right)& -T_2\le W_{in}<-T_1\\ W_{in}-\left(T_2\cdot \left(G-1\right)\right)+\left(T_1\cdot G\right)& W_{in}<-T_2\end{array},$$

(9)

where W_in and W_out are the wavelet coefficients before and after enhancement transfer, respectively; G is the gain, which is empirically selected within a range in a scene and weather condition; T₁ and T₂ are two thresholds with T₁ < T₂, and T₁ is determined by the image size n and the variance of the image σ [10]. The coefficients transferred by the function shown in Equation (9) will be adjusted by an appropriate gain to improve the clarity of the images. The appropriate gain is obtained by evaluating the corresponding clear image quality through iterative gain in the range. Therefore, with the separate processing of the low-spatial frequency and high-spatial frequency components, both the clarity and visibility of rainy-day images can be improved.

3. Results and Discussion

To demonstrate the practicability and performance of the proposed method based on the CP characteristics, we conducted several experiments under different weather conditions, and the results were analyzed.

3.1. Effect of Water Spray Removal

The experimental results for three different scenes are presented in the following discussions, and the corresponding weather conditions are also listed in

Table 1. Weather conditions of different scenes.

Parameters	Scene One	Scene Two	Scene Three
Temperature (°C)	20	21	25
Rainfall capacity (mm/h)	1.9	1	0.6
Relative humidity	99%	90%	55%
Wind strength (Km/h)	5	10	10
Wind direction	North	North	North
Rain level	Light rain	Light rain	Drizzle

. Firstly, using Scene One shown in Figure 5a as an example, this image was taken at the North Gate of Xidian University, Xi’an, with low contrast and weak visibility. After being processed by the image enhancement algorithm in reference [24], in Figure 5b, the outlines of the buildings become more distinct. However, the whole image is still shielded by a piece of gauze due to the scattering effect. In comparison, the image in Figure 5c processed by our method shows enhancement effects in not only contrast but also the details of the image. For instance, the building marked by the red rectangle in Figure 5b is over 1.2 km away from the camera and is hardly visible in Figure 5a,b, while Figure 5c provides a much clearer image of this building as well as the buildings around it. In addition, for the nearby building about 0.83 km away from the camera outlined by a green rectangle in Figure 5a, its contrast is sharply enhanced. Overall, the method proposed in this study works well to improve the contrast of rainy-day images and optimize water spray opaque regions to some extent in rainy-day images.

To demonstrate more quantitative data, a horizontal line of the intensity value versus the horizontal position of the image is plotted in Figure 5. Taking the vertical position of pixel 246 from the top, this horizontal line passes through buildings A, B, C, D, and E labeled in Figure 5c. The three shaded areas labeled by 1, 2, and 3 represent the background between A and B, B and C, and C and D, respectively. The variant trends of the blue and red curves are very similar, but the contrast of the image denoted by the blue curve is superior to that of the red one, which is improved by about 2.6 times. This is due to the removal of water spray scattering effects, which can highlight the buildings out of the background of the sky.

Experimental results for the second scene are exhibited in Figure 6. The original rainy-day image in Figure 6a exhibits low contrast, whereas Figure 6b presents a much better result processed by the proposed method. Detailed improvements can be noticed through the buildings labeled as A, B, and C. Firstly, the definition and visibility are significantly improved. Not only is the building outline much clearer, but also more details (such as floors of the building) emerge in the processed image. Secondly, the horizontal lines of vertical pixels 120 and 85 from the top pass through buildings A, B, and C, and cranes D and E in both the original rainy-day image and the reconstructed image by our method. Areas 1, 3, and 4 marked by the green rectangle clearly highlight the building and cranes A, D, and E in the reconstructed image, respectively. Furthermore, a small distance between buildings B and C is also significantly exhibited by the prominent peak of the red curve in the shaded area 2. As for contrast, the black curve presents a gentle variation trend, while the sharp concave peaks emerge from the red curve. In addition, the tower cranes show in Figure 6e,f also clearly display the changes (from invisible to visible). Compared with the original image, the processed images indeed make it easier to locate targets.

3.2. Color Analysis

One of the most straightforward characteristics of an image is color. In the following parts, the effects of the proposed method are demonstrated on the reconstruction of actual image color. The RGB color space model is chosen. The intensities of each pixel in channels R, G, and B are tallied. Taking Scene One as an example, the statistical results are given in Figure 7. The proposed method in Ref. [24] could improve the image contrast to some extent (Figure 7b), but the improvement in color is limited, since the extension of the intensity statistic distribution is limited, as exhibited in Figure 7e. In comparison, Figure 7f is considerably extended as well as the dynamic range of the image color. The color distribution of the original image ranges from 30 to 120. The clear image by our method is shown in Figure 7f, which has an extended dynamic range from 0 to 255. Additionally, the color distribution of our method is about twice that of Ref. [24].

In addition, Figure 8 presents the results for Scene Three. Figure 8b was processed by the proposed method, displaying higher contrast and a better visual effect than Figure 8a. Additionally, the dynamic range of color is noticeably improved by approximately 2 times that of the original image, indicating much clearer details of the scene.

All the results demonstrate the capabilities of CP imaging technology in the effective reconstruction and correction of color decay brought about by the scattering of water spray, as well as the enhancement of image quality.

4. Conclusions

This study presents a CP imaging method for enhancing the vision in rainy-conditions by combining the multi-scale analysis in the frequency domain. This method can enhance the contrast of rainy-day images and recover image color. It is based upon the interaction between light and water spray including absorbance and scattering. The key step is using the CP memory effect in the low-frequency parts to improve the quality of images in rainy conditions and compensate for the specific structure information by the high-frequency parts. By this method, much clearer images with better image colors can be reconstructed. This processing method provides a new way to remove the influences of water spray on images and enhance visibility on rainy days. For future work, we will extend this method to video monitoring such as a driving safety assistance device.