Automatic Suppression Method for Water Surface Glints Using a Division of Focal Plane Visible Polarimeter

To address the problem of water surface detection imaging equipment being susceptible to water surface glints, this study demonstrates a method called De-Glints for suppressing glints and obtaining clear underwater images using a division of focal plane (DoFP) polarimeter. Based on the principle of polarization imaging, the best polarization angle and the image corresponding to the minimal average gray level of each pixel are calculated. To evaluate the improvement in image quality, the index E was designed. The results of indoor and outdoor experiments show that the error of the angle calculation of this method is within 10%, and the minimum error is only 3%. The E index is positively improved and can be relatively improved by 8.00 under the interference of strong outdoor glints, and the method proposed in this paper shows a good adaptive ability to the dynamic scene.


Introduction
For human survival, a good water environment is indispensable. Effective management of the water environment requires regular and continuous surveillance of the surface environmental conditions of rivers, lakes, and seas in order to obtain the distribution of floating objects or aquatic organisms on the water surface, underwater plants, and pollutants discharged underwater. Sensors [1][2][3], hydrological remote sensing [4], imaging detection [5][6][7], and other technologies are effective technical means to achieve these ends. Imaging detection techniques provide intuitive water conditions in the form of non-contact telemetry images compared to sensor detection techniques and result in the better identification of large volumes of solid pollutants on the water surface. Compared with hydrological remote sensing, such approaches are more suitable for small-scale clear monitoring near the surface of the water. Different types of cameras have been employed as important monitoring tools. Some cameras are arranged on traditional water surface monitoring boats; meanwhile, along the river, fixed monitoring cameras or unmanned aerial vehicle dynamic cameras are increasingly becoming important means of monitoring the water for rapid and effective surveillance.
The monitoring of water environments is arduous because of the reflection of sunlight. On clear days, the sunlight reflected from the water surface often produces strong glints, which can cause large-area saturation of the camera, the loss of pixel information, the erroneous judgment of underwater target detection, and other strong radiation interference problems. In these cases, errors are often introduced in ocean water color remote sensing, surface target surveillance, and other imaging applications [8]. Therefore, eliminating or minimizing the influence of water surface glints significantly affects image quality [9][10][11].
Glints are produced by the reflection of strong sky or solar radiation, which exhibit remarkable polarization features [12,13], related to the observation and solar conditions [14][15][16][17]. It has long been common knowledge that glints can be reduced by using polarized imaging [18,19]. A polarizer is an essential accessory in photography [20]. Photographers can obtain polarized images by rotating the polarizer attached in front of the camera. Such systems are referred to as division of time (DOT) polarimeters [21]. When using DOT polarimeters, the transmission direction of the polarizer should be perpendicular to that of the glints. In this state, information on intensity can be obtained with minimal glint impact in a single polarization direction [22,23]. In line with DOT polarimeters, orthogonal polarimetric imaging modes have been developed to better suppress glints by capturing images in the best and the worst detection directions. Glint suppression methods based on DOT polarimeters have achieved remarkable results in static scenarios. However, in practical engineering applications, their development has been limited by their complex construction. Accurate and dynamic adjustments of position are challenging because of the constantly changing platform and detection position. This renders the dynamic optimization of the direction of the best detection angle necessary. Moreover, the relative inclination of the water surface within the detection field of view is also inconsistent.
These issues have been well addressed since the introduction of the fully polarized imaging technique. Fully polarized imaging systems can simultaneously obtain polarization images of the scene in three to four directions, which can be used to not only obtain the image of any detection direction of the scene, but also calculate polarization information such as the degree of polarization (DoP), angle of polarization (AoP), and Stokes vector. Typical polarization imaging modes such as the division of amplitude [24], division of aperture [25], and DoFP polarimeters [26,27] have been proposed and developed extensively, among which the DoFP polarimeter is the most compact. It couples an array of 2 × 2 micro polarizers as a polarization unit to the corresponding CMOS detector image pixels. Thus, the intensity of the four detection directions can be obtained dynamically. The DoFP polarimeter is a current research hotspot and has made important advances in applications such as food testing [28] and biomedical applications [29]. It is expected to achieve glint suppression by individually calculating polarization information according to the fluctuation of the water surface in different areas, thus rendering it particularly suitable for environmental detection in scenes containing water surfaces or shallow water areas. Meanwhile, the DoFP polarimeter retains the advantages of the DOT polarimeter in suppressing water surface glints, while featuring real-time performance. However, there is still insufficient research on how to quickly achieve glint suppression based on real-time polarization images.
In this study, the dynamic adaptation capability of the DoFP polarimeter is utilized to study the automatic suppression method of water surface glints in water environmental observation. Section 2 of the paper presents a novel polarization-based suppression method for a DoFP polarimeter based on a Stokes vector polarization imaging model. Furthermore, Section 3 introduces the experiments and Section 4 presents a comparison of the proposed method using DoFP with previous methods using DoT polarimeters, before finally analyzing the results.

Methods
Based on the Stokes vector polarization imaging model, we analyzed the conventional method by rotating a single polarizer and studied the polarization De-Glints method using DoFP-visible polarimeters in this section.

Principles of Polarization Imaging
Polarized light is generally described by the Stokes vector S, which can be expressed as where S 0 is the total light intensity, S 1 is the horizontal or vertical linear polarization component, S 2 is the linear polarization component of +45°or −45°, and S 3 is the left-or right-handed circular polarization component.
When examining the polarization state of a scene, two parameters are typically used: the DoP and AoP. The DoP represents the proportion of the polarization component in the total intensity. The AoP represents the angle between the direction of the maximum polarization energy in the incident light and the x-axis of the reference coordinate system. Normally, the circular polarization condition (S 3 = 0) is rarely satisfied when light is scattered in the atmosphere or reflected from a water surface [30]. Supposing S 3 = 0, according to the Stokes vector S, the equation for calculating DoP and AoP is expressed as The Stokes vector of arbitrary light can be represented by DoP and AoP as follows: The change in the polarization state caused by the optical element of the light wave is described by Mueller matrix M: where S in is the incident Stokes vector, S out is the output Stokes vector, and M is the Mueller matrix of an optical element that represents the effect of an optical element or system on the incident light.
The photoelectric imaging device can only respond to light intensity; thus, the total light intensity is expressed as The Stokes vector of incident radiation can be calculated according to the light intensity values measured in four uncorrelated directions. Subsequently, the polarization information of the incident radiation, such as the DoP and AoP, can be obtained.

Traditional Glint Suppression Method by Polarization
The light reflected from the water surface exhibits high intensity and obvious polarization characteristics. The objects on or under the water usually have low intensity without polarization characteristics in the natural scenes. We denote the light with and without polarization characteristics as I p and I up , respectively. Thus, the original image I can be represented as According to Marius's law, the intensity of the incident light after passing through the polarizer becomes where ∂ is the vibration direction of the linear polarization light and θ is the transmission direction of the linear polarizer. The installation of a polarizer in front of the lens, with the transmission direction being orthogonal to the vibration of the linear polarized light, can eliminate the reflected polarized light and suppress glints [18]. In the experiment, the observation geometry was determined according to a theoretical model and the glint suppression direction was determined by rotating the polarizer [19]. To reduce the error caused by the manual rotation of the polarizer and to increase the speed, a stepper motor can be used [31]. Consequently, the polarization differential method was implemented. This method calculates DoP through two orthogonal polarized images: The polarization differential method combined with the scattering model has been widely used for image defogging and underwater de-scattering [32][33][34].

Automatic Glint Suppression Method Based on the DoFP-Visible Polarimeter
The DoFP polarimeter can simultaneously acquire images in four different polarization directions (0°, 45°, 90°, and 135°) of the scene. This is conducive to obtaining the polarization images of the dynamic scene in real time, calculating different parts of the image independently, solving the Stokes vector, and calculating the DoP and AoP. These outstanding advantages aid the DoFP polarimeter in overcoming the inherent defects of the DoT polarimeter.
The Stokes vectors S 0 , S 1 , and S 2 can be expressed as The general expression for the Mueller matrix M p of an ideal linear polarizer with a theoretical angle θ between the transmission and horizontal direction (x-axis) is expressed as follows: cos 2θ sin 2θ 0 cos 2θ cos 2 2θ sin 2θ cos 2θ 0 sin 2θ sin 2θ cos 2θ As the first row of M p is M m , combining this with Equation (5) yields the light intensity after transmission through the polarizer as The images in each polarization direction can be derived using Equation (11). Combined with Equation (7), it is evident that the detection light intensity I θ was related to the transmission direction θ of linearly polarized light. When θ was perpendicular to the vibration direction ∂ of polarized light, the polarization part was reduced to 0, the detection light intensity reached the minimum value and the glint was suppressed to the greatest extent.
The grayscale value of the image in the direction θ is f θ (i, j). We define the average grayscale f θ as the index for evaluating the effect of the glints on f θ (i, j): where WH refers to the image size. If the image in the direction θ i has the minimal average grayscale f θi , θ i is the best polarization angle θ best wherein the glints exert the minimum effect on the image.
Therefore, the image at the best angle after polarization processing can be expressed as follows (the DoFP-14 method in brief): In practice, the water surface is not perfectly still owing to the influence of moving objects and ambient winds. Thus, the polarization characteristics of each part are not the same. To obtain a better image effect, each pixel was solved via polarization. The Stokes vector at (i, j) is S(i, j) and the corresponding Muller matrix is M m(i,j) ; then, the light intensity after polarization processing is expressed as follows (the De-Glints method for short):

Experiment
The aim of this study is to compare the effectiveness of glint suppression between DoT and DoFP systems. We use a PHX050S DoFP polarimeter (LUCID Vision Labs) with a sensor size of 2048 × 2448 px. The focal length is 25 cm. The image format adopted is 8 bits, and the max extinction ratio is 425:1. Simultaneously, a CM3-U3-13S2M monochrome ordinary camera with a sensor size of 960 × 1280 px is used to acquire polarized images with rotating polarizers as a DoT system. The polarizer extinction ratio is 800:1. The focal length is 12 cm. We record data on light intensity at the water surface using a hand-held digital light meter (Aicevoos AS-V10 Digital Light Meter) with a range of 0 to 200,000 lux. Its resolution is 0.1 lux for illuminances less than 1000 lux and 1 lux for illuminances larger than or equal to 1000 lux.
The DoFP system possesses a significant advantage in dynamic processing, rendering it the focal point of this study. However, the structure of its detector leads to crosstalk among adjacent pixels, which ultimately dilutes its accuracy in detecting polarization information when compared with the DoT system. Therefore, we investigate the accuracy, effectiveness, and adaptability of the DoFP system within the laboratory and in field scenarios.
The experimental scenario and equipment are shown in Figure 1. We perform nine sets of indoor experiments including six static scenes T1-T6 and three dynamic scenes V1-V3. Static experiments are used to verify accuracy and effectiveness. Dynamic experiments are used to verify adaptability. The details of the experiments are shown in Table 1. The light source for indoor experiments is a conventional white LED, which is placed in a highly reflective softbox for uniform illumination when in use. The water depth of the object and the light intensity on the water surface are recorded. The inclination of the light source for all nine experiments is 68°and the exposure time is 50 ms. The observation elevation angle indicates the angle between the camera's optical axis and the water surface in the horizontal direction. We simulate static scenes and different dynamic scenes by controlling the wind to disturb the water surface at different speeds.  Outdoor experiments O1-O3 in three different locations are conducted in Shichahai, Beijing, to verify the practical utility, as shown in Figure 1b and Table 1. We regard the tank under the water where the lotus is cultivated as the main observation target. The water surface is constantly disturbed by winds from different directions.

Results and Discussion
To effectively handle dynamic scenes and overcome the limitations of the DoT system, the proposed DoFP-polarimeter-based glint suppression method is experimentally verified and discussed.

Criteria
To facilitate objective evaluation, the following indexes are used: 1.
The accuracy of the angle calculation is described using the relative error δ where θ m is the angle calculated from the data obtained via the DoFP polarimeter and θ 0 is the angle obtained via the DoT polarimeter, which is considered to be the standard value.

2.
We use the comprehensive image quality evaluation index E to evaluate image quality: where C is the contrast, G is the mean gradient, and σ is the standard deviation. Contrast is defined as the intensity ratio of the target to the background: where µ t and µ b represent the average intensities of the target and local background around the target, respectively. The contrast between the target and background reflects the enhancement of the target. A higher contrast indicates a smaller impact of the glints and a better effect of the algorithm. Average gradient refers to the variation in the image detail grayscale values. A larger average gradient indicates richer detail.
The standard deviation reflects the discrete condition of the image grayscale values: The original image is affected by the glints and has a high local grayscale value, which manifested as a large standard deviation. The extreme distribution of pixel grayscale values should be reduced after processing; therefore, the smaller the standard deviation, the better the method. The E indicator is proportional to the contrast C and average gradient G and inversely proportional to the standard deviation σ. Combined with the aforementioned analysis of the three indicators, the larger the value of E, the better the quality of the characterization image.

3.
The relative improvement in the E index, δE, is used as an indicator to evaluate the effectiveness of the algorithm: where E p and E o are the E indicator values of the processed and original images, respectively.

Comparison of DoT and DoFP Systems
As shown in Figure 2, to verify the accuracy of the calculations, six sets of static experiments are performed indoors. Figure 2a shows the original images obtained using the DoT polarimeter. By rotating the polarizer of the DoT system, we obtained the images with minimal intensity as the polarization suppression results (shown in Figure 2b). As is evident, the glints are effectively suppressed after filtering by the polarizer, and the spots of the original image with a high gray value mostly disappeared. The difference in intensity between different parts of the target (above or below the water surface) in T1 and T2 is reduced and the image is more integral. Furthermore, the underwater objects in T3-T6 are clearer. The original images obtained by the DoFP polarimeter are shown in Figure 2c. The results of calculation using the DoFP-14 method are shown in Figure 2d. The glints are effectively suppressed and the details above and below the water surface are clearer. Regardless of a DoT or DoFP polarimeter, the images after polarization processing are all sharper than the original images.
The best polarization angles measured by the DoT polarimeter and calculated by DoFP-14 for the six scenes are presented in Table 2, respectively. The calculated results are different from the polarization angle obtained by rotating the polarizer. In fact, there are certain errors between the DoT measurements and the actual true values, and the relative errors here are only calculated to compare the differences between the DoFP and the DoT polarimeters. The data in the table show that of the six comparison scenes we selected, the deviation of the T2 group is the largest, and the relative errors of the remaining groups are within 10%, indicating that the best polarization angles calculated by the DoFP system and the DoT system are basically the same.

Static Results
Considering that the polarization direction is not the same everywhere on the water surface, we suppress the glints using De-Glints; the results are shown in Figure 2e. In the previous section, we discussed the remarkable effect of the polarization method in Figure 2a-d. Figure 2e also clearly shows that the processing results effectively suppressed the glints. However, it is difficult to subjectively judge whether there is an improvement compared to the previous method. The comparison is made through the evaluation indicators proposed in Section 4.1.
We calculate the E indices for a-e in T1-T6, as shown in Figure 2, and obtain the relative degree of enhancement δE after processing, which represents the degree of improvement in image quality. δE 1 , δE 2 , and δE 3 are the degrees of E index enhancement after DoT, DoFP-14, and De-Glints polarization processing, respectively. The results are presented in Table 3. As is evident, for all six groups of the experiment, the values of δE 3 are almost all bigger than δE 1 and δE 2 , indicating that the results for De-Glints are better than those for DoFP-14 and DoT, which means that the De-Glints method is more effective.

Dynamic Results
According to the previous analysis, the DoT polarimeter is structurally destined to be inferior to the DoFP polarimeter for dynamic scenes. In this section, we perform three sets of dynamic experiments V1-V3 with different wind speeds and analyze the change in the best polarization direction. We judge the image quality using E as an evaluation index to verify the dynamic adaptive capability of the DoFP system.
The results are shown in Figure 3. Figure 3a shows the original images obtained by the DoT polarimeter. The DoT system cannot change the polarization direction of the polarizer in real time. Under the same illumination and geometric conditions, water surface disturbances can affect the polarization characteristics, which may change the best polarization direction. From Figure 3b, we can see there is also significant glint suppression in one fixed polarization direction, because of the attenuation of the light intensity by the polarizer. The polarized images showed that the surface reflections are suppressed and underwater targets are more clearly defined. The original images acquired by the DoFP polarimeter are shown in Figure 3c. Ten consecutive images of the dynamic scene were processed using DoFP-14 and De-Glints, as shown in Figure 3d,e, respectively. The glint is well suppressed and the image is considerably clearer. The variation in the best polarization angle calculated using DoFP-14 is shown in Figure 4. As is evident, in dynamic scenes, the best polarization angle fluctuates within a range with the water surface. With the increase in the wind speed, the water surface undulates faster and more dramatically, and the relative change in the best polarization angle increases. Furthermore, the DoT polarimeter is affected to a large extent by the changes in water surface conditions, which is difficult to overcome. For the DoFP system, the best polarization angle can be calculated separately for each frame in real time, such that the DoFP-14 and De-Glints are not affected by changes in the best polarization direction. In dynamic scenes, the DoFP polarimeter exhibits outstanding superiority.
The E indices of the original, DoFP-14, and De-Glints images of the water surface in V1-V3 are shown in Figure 5. After polarization processing, the E index increased significantly, representing a significant improvement in image quality. The results of DoFP-14 and De-Glints are close, but those of De-Glints are slightly better.

Outdoor Results to Verify Practical Utility
As shown in Figure 6, we chose three different locations for our experiments, O1-O3. It is evident that, in the actual scene, the glint has a strong interference with the underwater target, blurring the details of the target and introducing interference, or limiting the integration time of the image to prevent overexposure owing to the high intensity of the glint. This results in low gray values of the target, which are difficult to distinguish for the human eye. In the De-Glints results, the effect of the glints is reduced. The comparison in Figure 6 shows that the glints are largely suppressed and the underwater scenes are displayed more clearly after De-Glints processing. In the case of very heavy surface flare effects (Figure 6c), the target details in the water in the original image are completely drowned out. After processing, the glint is well suppressed and the underwater target is faintly visible. The evaluation indicators of the images in Figure 6 are shown in Table 4. The E indicators of the De-Glints results are all positively enhanced. In the extreme case of O3, the δE is 8.00. This result shows that the information about water clutter and glints in the processed images is suppressed and that the underwater targets are enhanced, proving the effectiveness of the method in practical scenarios.
As shown in Figure 2b, the DoT polarimeter also shows an effective performance in glint suppression. In outdoor experiments, however, it shows obvious limitations. The original image obtained using the DoT polarimeter in O1 is shown in Figure 7a. The polarization image is shown in Figure 7b. We mainly wish to discuss two points of interest, marked by red rectangles in Figure 7c. In polarized images, the glint is unevenly suppressed in each region. In rectangle 1, we see that the residual glint is very obvious in the marginal region, whereas it is suppressed well in the DoFP results, as shown in Figure 7a. During the experiment, when we rotated the filter to find the best polarization direction, the position of the floating object on the water surface in rectangle 2 moved significantly, which was a situation in the DoFP system that could be comfortably avoided.

Enhancement
Whether it is a DoT or DoFP system, after polarization detection, the polarized image will be darker than the intensity image. Image-enhancement techniques can improve the image intensity level and distribution, and help the subjective judgment of image quality. We take the histogram equalization method as an example. Figure 8 shows the enhanced original and polarized images in O1-O3, respectively. As evident from the figures, the effect of water surface glints in the original image is amplified following processing with the superimposed enhancement algorithm, which seriously affected the observation of underwater scenes. For the polarized image, the targets are well enhanced. In the case of very heavy surface flare effects (Figure 8c), the underwater target details in the original image are completely drowned out. Following histogram equalization, glints and ripples are magnified and the underwater target is completely invisible. After De-Glints processing, the glint is well suppressed and the underwater target is faintly visible, and then after equalization, the outline of the underwater object is clearly shown. Thus, polarization imaging is very effective in the suppression of surface glint and makes a significant contribution to subsequent enhancement algorithms, even recognition algorithms.  However, certain areas of this study require improvement, and the accuracy of the polarization direction calculation and the overall homogeneity of the results will be further optimized in the future. The method described in this study is based on the principle of polarization imaging, and the intensity of the results after polarization filtering is generally low. Thus, the superimposed histogram equalization method in this study is only a small attempt to visualize the results, and how better to design the algorithm is something that needs to be studied.

Conclusions
In this study, we present De-Glints, a method for suppressing solar glints and obtaining clear underwater images through the water surface based on a DoFP-visible polarimeter. This method is dependent on the significant partial polarization characteristics of the glints. The DoFP-visible polarimeter is used to obtain the characteristics of different polarization directions simultaneously, perform polarization measurements on the water surface disturbed by the glints, create polarization radiation maps in four polarization directions, and calculate the Stokes parameters. Based on the principle of polarization imaging, the best polarization angle and the image corresponding to the minimum average grayscale of each pixel are calculated to achieve the suppression of glints. In this study, index E is designed to evaluate the degree of image quality improvement. We examine the accuracy, effectiveness, and adaptability of the proposed method using the DoFP system while also validating its practical utility in real-world scenarios.
The results of indoor and outdoor experiments show that the error in the selection of the best polarization angle using DoFT calculations is essentially within 10% compared to the DoT polarimeter. The minimum error is only 3%. The proposed methods, DoFP-14 and De-Glints, improve the image quality significantly in both evaluation indices, E; however, De-Glints works a little better. The DoFP polarimeter demonstrates significant superiority in terms of adapting to dynamic scenes. In addition, the results of the outfield experiment also prove that, in the case of strong flare interference, the glint is effectively suppressed following polarization processing, and the E index could also be relatively improved by 8.00. The details and contour information of the target in the water body are clearer, which proves the correctness and necessity of the method in this study.
The proposed method can be applied to improve the detection, identification, and tracking of targets in water. Thus, using the target polarization information to effectively suppress flare interference and improve the contrast of the target has important practical application significance.