Dehazing with Offset Correction and a Weighted Residual Map

: In hazy environments, image quality is degraded by haze and the degraded photos have reduced visibility, making the less vivid and visually attractive. This paper proposes a method for recovering image information from a single hazy image. The dark channel prior algorithm tends to underestimate the transmission of bright areas. To address this problem, an improved dehazing algorithm is proposed in this paper. Assuming that intensity in a dark channel affected by haze produces the same offset, the expected value of the dark channel of a hazy image is used as an approximation of this offset to correct the transmission. However, this correction may neglect scene difference and affect the clarity of the recovered images. Therefore, a weighted residual map is used to enhance contrast and recover more information. Experimental results demonstrate that our algorithm can effectively lessen color oversaturation and restore images with enhanced details. This algorithm provides a more accurate transmission estimation method that can be used with a weighted residual map to eliminate haze and improve contrast.


Introduction
Aerosols in the air scatter light into the atmosphere. This scattering impairs the direct transmission of scene radiance and degrades image quality, especially on hazy days [1]. These degraded images usually have low contrast and saturation, loss of detail, and hue shift, thereby affecting visual effects and subsequently image processing. Hence, many approaches have been developed to eliminate haze and generate realistic clear images.
Based on the atmospheric scattering model describing the attenuation and distribution of light through aerosols, a hazy image is described as a convex combination of scene radiance and atmospheric light, and the coefficients of this equation are determined by the scene transmission of each pixel in the image. In a color (RGB) image, each pixel of the model has four unknowns, the scene radiance per color channel (one each for R, G, and B) and a transmission value. However, a single image can only provide three constraints for each pixel, i.e., the intensities of the three channels. Therefore, more constraints are needed to address this uncertainty. Some methods use additional information about the scene, such as multiple images taken under diverse conditions [2], polarization angle [3], or geometric features of the scene [4], to determine transmission and obtain haze-free images.
More recently, some dehazing methods using a single image have been developed; for example, by assuming that local transmission is not correlated with surface shading [5], relaxing the physical residual map is used to further improve the contrast of recovered images. In Section 3, we present the results of our proposed algorithm for hazy images; conclusions are presented in Section 4.

Image Degradation Model and Dark Prior
According to the atmospheric scattering model [33], images captured in hazy weather can be expressed as where I(x) is the observed intensity; J(x) is the radiance of the real scene, a haze-free image that needs to be recovered; describes the portion of the radiance that is not scattered during propagation through the atmosphere to the imaging system, where β is the atmospheric scattering coefficient related to the wavelength, and d(x) is the scene depth. The atmospheric light A represents the intensity of ambient light and is usually approximated as a global constant. Analyzing Equation (1), when the transmission and atmospheric light are known, a dehazed image can be obtained as follows: Therefore, based on this model, the key for the dehazing methods is to obtain the transmission and atmospheric light of the scene.
To estimate the transmission, He et al. [8] used the dark channel prior (DCP), statistics of the locally minimized images (dark channel) of clear images to compute the transmission and restore the image. The dark channel is defined as where J dark (x) is the dark channel of J(x), J C (x) is a color channel of J(x), Ω(x) is the neighborhood of x, and its size is 15 × 15. He et al. [8] believed that 0.1% of the brightest pixels in the dark channel usually belong to most haze-opaque regions. Picking the brightest color in these pixels is equivalent to choosing the largest intensity in the area with the heaviest haze, which is the optimal solution for the atmospheric light A. This paper uses the same method to estimate A of each channel as that in [8] .
Minimizing Equation (2), the transmission can be estimated by According to the dark channel prior, for most local areas that do not cover the sky, there are always pixels with at least one color (RGB) channel with very low radiance intensity, i.e., J dark (x) → 0. Then, the transmission can be generated as follows: The dark prior is valid and effective when there are shadows, colorful objects, or surfaces of objects and dark objects or surfaces in the scene. However, when there is an extensive bright region in the hazy image such as whitish sky or objects, this prior will lead to underestimation of transmission, which will result in color cast or oversaturation in these regions of recovered images.

Transmission Estimation Using Offset Correction
To solve this problem, this study devises an improved algorithm based on an offset correction to more accurately estimate the transmission of bright regions.
Considering that the dark channel of haze-free images in the bright area may have a high value, the dark channel prior is extended to a full scene prior: when α(x) = 0, it is a dark channel prior, and when α(x) ∈ (0, 1), it is a prior satisfying bright scenes. Due to the variability of the scene, the statistics α(x) of the bright regions that do not conform the dark channel prior are difficult to calculate quantitatively and can only be approximated.
To reasonably estimate α(x), this paper proposes a method to estimate transmission using the expected value of the dark channel of foggy images. It is known from the dark channel prior that the expected value of the dark channel image of clear outdoor images is zero. For hazy images, the intensity of the dark channel affected by the haze will increase on the whole. Correspondingly, its expected value will produce an offset compared with clear images.
In this study, we assume that the brightness of the dark channel in the scene is affected by haze to the same degree and has the same increment. Then, the statistics α(x) can be approximated by the difference between the dark channel I dark (x) of a hazy image and its expected value . To ensure where max (•) is the maximum function, I dark (x) is the dark channel of the hazy image I(x), and µ cen is calculated as where MN denotes the number of pixels. Bringing Equation (6) into Equation (4) gives an offset-corrected estimation of transmission: when the offset-corrected transmission t(x) is calculated, according to Equation (2) the dehazed image J(x) can be estimated as We neglect the difference in the degree of impact, which may affect the clarity of a dehazed image. Fortunately, we can improve the contrast using a weighted residual map, which will be discussed next.
Although the expected value is only an approximation, this approach can effectively reduce the estimation error of the transmission in bright regions and avoid color oversaturation. Figure 1 shows a comparison between DCP and our method with regard to transmission and the recovered image. As shown in Figure 1d, our method can more accurately estimate the transmission of bright regions and exhibit better transition at the edges. Unlike the setting of the minimum threshold for the transmission of bright areas, this method does not just reflect the uniqueness of shooting conditions, such as scene and illumination, but can also consider the intensity difference in bright areas to obtain an estimated value that is closer to the true transmission value.

A Weighted Residual Map
As the expected value is a measure of the central tendency of the intensity of the dark channel image, it cannot accurately calculate the transmission of each pixel and thoroughly remove haze in regions where local intensities in the dark channel are significantly higher than the expected value. Therefore, we propose a dehazing approach to further recover image details using a weighted residual map.
The fundamental idea of a weighted residual map is to use a residual map between the actual observed image I(x) and the image J(x) estimated by the offset-correcting approach for dehazing. The residual map R residual (x) is defined as As shown in Figure 2a, this residual map contains unrecovered scene and structure information that we can use to estimate a more accurate recovery of the original image.
Substituting Equation (10) into Equation (11) gives From Equation (12), the residual map R residual (x) is proportional to the offset-corrected transmission is negatively correlated with transmission, as is the offset-corrected transmission t(x). From Equation (12), the residual map R residual (x) is also negatively correlated with the offset-corrected transmission t(x). Therefore, according to Equation (12), regions in the residual map with lower transmission that are farther away from the imaging device have larger intensity magnification, which is the opposite of what we desire. Therefore, we apply a negative residual map for contrast enhancement and get a dehazed image as follows: where ω is the weight function, which is used to further adjust the influence of different regions in the residual image on the final restored image J(x). In this paper, the offset-corrected transmission t(x) is chosen as ω: The appealing property of this selection lies in converting an inverse function to a linear function while preserving distance dependence. Chosing t(x) as ω makes these two multiplicative terms in Equation (14) of the same order of magnitude, thus avoiding overamplification, and the range of the value can be limited to [0, 255]. Besides, this selection will balance the restored information and excessive sharpening, as shown in Figure 2b.
The pseudo code of the dehazing algorithm with offset correction and a weighted residual map is in Table 1. Firstly, the offset-corrected transmission t(x) is estimated by calculating the full scene prior α. Secondly, the final dehazed image J(x) is estimated according to the dehazed image J(x) by the offset-correcting approach and the weighted residual map t(x)R residual (x). Table 1. Dehazing algorithm implementation with offset correction and a weighted residual map.

Parameters:
Hazy image: I(x), Atmospheric light: A, Number of pixels: MN Procedures: • Estimation of transmission -Calculate the dark channel I dark (x): I dark (x) = min c∈{r,g,b} min y∈Ω(x) I C (y) -Calculate the full scene prior α(x):   Figure 1c,e, the quality of the recovered image in Figure 2c, which has clearer edges and more natural colors, is further improved by this residual-based approach.

Results and Discussion
In this section, we conduct various experiments on both real-world images and the synthetic dataset to demonstrate the effectiveness of the proposed method. Moreover, we do an ablation study to demonstrate the effectiveness of our offset correction and the weighted residual map.

Ablation Study
To better reveal the effectiveness of offset correction in transmission estimation and the weighted residual map, we conduct an ablation study on images with or without sky regions. We compare our apprach with DCPin Figures 3 and 4. In Figure 3, due to underestimating the transmission of extensive bright regions, the results of DCP show oversaturation and artifacts in sky regions in Figure 3c. Figure 3d,e demonstrate that our offset correction method works well in these regions.
In Figure 4, although there is no significant difference between the transmission estimation of DCP in Figure 4b and our transmission estimation in Figure 4d for image without sky regions, the weighted residual map can further enhance final results by capturing fine structural details. As shown in Figures 3  and 4e,g, we observe that whether there are whiteish regions or not, the weighted residual map can take effect on improving image clarity.
The ablation study demonstrates that offset correction in transmission estimation and the weighted residual map are effective for image dehazing.

Performance on Real-World Hazy Images
To display the performance of offset correction and a weighed residual map on haze removal, we apply our dehazing algorithm on both real-world hazy "Florence", "Community", and "Buildings" images containing a large grayish white sky or objects and real-world hazy "Flags", "Forest" and "Girls" images whithout sky regions.
In Figure 5, we compare the proposed method with the algorithms proposed by DCP, Berman et al. [13], Bui et al. [17], DCPDN [29], Zhu et al. [16] and Golts et al. [18]. In DCP, the colors of restored images are often oversaturated, especially in the sky areas, as the transmission is underestimated. Berman et al.'s results contain some saturated or dark pixels, and haze may not be sufficiently removed in some regions, for instance, in the buildings in the lower left corner of "Community" and "Buildings".

Performance on Synthetic Dataset
To further illustrate the effectiveness of the proposed algorithm, we use the large-scale RESIDE (REalistic Single Image DEhazing) test dataset [22] which contains 1000 synthetic images to test dehazing effects. The RESIDE test dataset is divided into indoor and outdoor parts, called "SOTS-indoor" and "SOTS-outdoor". In addition, we use the Peak Signal to Noise Ratio (PSNR) and the Mean Squared Error (MSE) to evaluate dehazed images. The PSNR of restored images is calculated to show objective evaluation results. The PSNR is obtained by calculating the logarithmic function of the MSE of two images to measure the image error. The larger the PSNR, the smaller the distortion. The subjective quality assessment test was conducted among thirty subjects, most of them were college students. The subjects were asked to assign each image a score between 1 and 5. 5 represents the best quality and 1 the worst. The 30 scores of each image were averaged to a final MOS of the image.
From Figures 6-11, it can be observed that even though DCP, Berman et al.'s [13] method and Zhu et al.'s [16] method, which are based on the atmospheric scattering model such as ours, can recover dehazed images, they tend to lead to either saturation or color shifting. It can also be found that DCPDN [29] and Golts et al.'s [18] method which are learning-based method are able to recover images with more natural colors. However, DCPDN tends to generate overexposed images, Golts et al.'s [18] method cannot fully remove heavier haze. In contrast, it can be observed from our results that they preserve more details and sharper contours. The comparison results are shown in Tables 2 and 3 in which the values are PSNR and MOS of the defogged images from Figures 6-11, respectively. The bold fonts indicate the optimal value of each set of results. Table 4 shows the average PSNR of the dehazed images of the RESIDE test dataset. The quantitative results of each part are shown in Figure 12. From Tables 2-4 and Figure 12, it can be observed that our results have higher PSNR and MOS, which indicates that our method performs well on subjective visual effects and can recover more information and edges from hazy images, and that our resulting images have a higher contrast.

Conclusions
In this paper, we proposed an improved dehazing algorithm based on the dark channel prior. This haze removal algorithm adapts an offset-correcting scheme, which is built on the assumption that the intensity of the dark channel in bad weather has the same increment owing to the haze. This is achieved by approximating the increment with the expected value of this dark channel to update the transmission in bright regions. Subsequently, a weighted residual map is introduced to recover more details and improve contrast. The results of our method show that a combination of offset correction and residual map does not just reduce color oversaturation but also enhances details in dehazed images. Besides having practical effects, from the perspective of image understanding, this method focuses on the residual information in images and uses it to improve the quality of recovered images.
Similar to other dehazing algorithms using the atmospheric scattering model, our method also suffers from the drawback of color shifting in defogged images. More advanced methods [34] focus on color shifting and preserve color information. In the future, we intend to investigate dehazing models and color balance algorithms [35] to address color shifting.