Adaptive Tolerance Dehazing Algorithm Based on Dark Channel Prior

: The tolerance mechanism based on dark channel prior (DCP) of a single image dehazing algorithm is less effective when there are large areas of the bright region in the hazy image because it cannot obtain the tolerance adaptively according to the characteristics of the image. It will lead to insufficient improvement of the transmission of image, so it is difficult to eliminate the color distortion and block effects in the restored image completely. Moreover, when a dense haze area or a third-party direct light source (sunlight, headlights and reflected glare) is misjudged as sky area, the use of tolerance will cause an inferior dehazing effect such as details lost. Regarding the issue above, this paper proposes an adaptive tolerance estimation algorithm. The tolerance is obtained according to the statistic characteristics of each image to make the estimation of transmission more accurately. The experimental results show that the proposed algorithm not only maintains high operational efficiency but also effectively compensates for the defects of the dark channel prior to some scenes. The proposed algorithm can effectively solve the problem of color distortion recovered by the DCP method in the bright regions of the image.


Introduction
Haze is a common natural phenomenon. Even on a clear summer day, distant targets are affected by haze due to the evaporation of surface water vapor. In weather conditions such as haze, the horizontal visibility [1] is significantly reduced due to the scattering of a large number of tiny droplets or aerosols suspended in the atmosphere. This change creates considerable difficulties in outdoor monitoring, automatic navigation, target tracking, etc., resulting in outdoor vision systems working improperly. Therefore, it is very important to study how to effectively treat the degraded images obtained in severe weather conditions. The processing of hazy images generally falls into two categories: enhancement-based algorithms and physics-based algorithms. Histogram equalization [2], homomorphic filtering [3] and Retinex [4] are all based on image enhancement algorithms. These algorithms can effectively improve the contrast of hazy images and highlight certain information in the image, but they do not consider the formation principle of hazy images and the degradation mechanism of the images. Another category is based on the atmospheric scattering model [5]. By analyzing the inverse process of the image degradation process and obtaining the relevant parameters of the degradation process, a clear image is recovered. The image restored by this method is real, and the image information can be more completely preserved. Therefore, a single image dehazing technology based on physical models has gained broad attention. Fattal [6] proposed removing haze from color images based on independent component analysis (ICA). However, ICA is ineffective for thicker hazy images due to the lack of color information. Tarel [7] used a median filter to calculate the minimum color component of the atmospheric veil. The median filtering of this method does not preserve the edges well, and the desired results cannot be obtained at deep discontinuous edges. He et al. [8] proposed a dark channel prior, first calculating the rough transmission according to the atmospheric scattering model and the dark channel prior and then using the soft matting algorithm [9] to optimize the transmission to obtain haze-free images. To solve the issues of edge loss and halo artifacts in haze-free images, Dilbag et al. proposed a series of DCP-based methods to enhance the estimation of atmospheric light, such as a modified joint trilateral filter (MJTF)-based DCP method [10], a modified gain intervention filterbased DCP method [11], and a fourth-order partial differential equation-based trilateral filter (FPDETF) dehazing method [12]. To reduce the color distortion, the DCP restoration model was also redefined. Recently, with the continuous development of deep learning, an increasing number of neural network algorithms have been used in the field of image processing and have achieved good results. Therefore, algorithms based on deep learning can be considered the third kind of dehazing algorithm. The existing learning-based dehazing algorithms mostly use deep learning to intelligently learn the haze feature and output the medium transmission map, and then recover the haze-free image through the atmospheric scattering model. For example, Cai et al. [13] proposed a trainable end-to-end system, DehazeNet, for medium transmission estimation. Li et al. [14] proposed a dehazing algorithm based on residual depth CNN.
Among the above algorithms, the DCP method proposed by He [8] has been extensively studied due to its simple principle and superior results. However, this method has two drawbacks. (1) The soft matting algorithm [9] needs to perform iterative calculations, which leads to high algorithmic complexity. To improve the computational efficiency, He [15] proposed a guided filter with edgemaintaining characteristics instead of the soft matting algorithm to refine the transmission map. However, the edge smoothing technique of local filtering suffers from halo artifacts, so the improved algorithms [16][17][18][19][20][21] were proposed for this problem. (2) If the image scene contains a large area of the sky or a bright white object, the dark channel prior will invalid, resulting in severe distortion of the restored image. To improve this phenomenon, Wang et al. [22] estimated the transmission map of the sky and non-sky areas and then combined it with the refined transmission map to remove the haze. Although the visibility of the sky area can be improved, this approach generally reduces the recovery performance among adjacent boundaries. Liu et al. [23] proposed a large sky region detection algorithm based on SVM classification, which uses two different strategies to obtain more accurate atmospheric light according to its detection results. Finally, the multiscale open dark channel model is used to adaptively calculate the dark channel for dehazing. Zhang et al. [24] proposed saliency prior to hazy imaging, which can distinguish white objects from dense haze by saliency detection. On the basis of the saliency prior, both an accurate airlight and a correct transmission map can be obtained from images containing large white objects, and finally, these images can be restored successfully. In addition, some authors have proposed tolerance mechanisms to correct the transmission of false estimates for such bright regions. The tolerance mechanism mainly refers to using this as a threshold to judge whether there is a bright area in the haze image and then performing segmentation processing on the sky and other areas of the image to achieve accurate correction of the transmittance image [25]. However, the tolerance values in their algorithms are generally the best or empirical values obtained by trial and error. In this paper, an adaptive tolerance mechanism is studied to adaptively determine the tolerance value according to the characteristics of the image to achieve the best dehazing effect. The simulation results showed that the improved algorithm greatly improves the dehazing performance. Not only is the restored image clear and natural, but the useful information in the image is not lost.
The remainder of this paper is organized as follows. Section 2 first introduces the dark channel prior dehazing algorithm and its defects. Then, in section 2.4, we describe the tolerance mechanism algorithm to solve the defect of the dark channel prior and its deficiency. In Section 3, we describe the details of the improved algorithm in this paper. In Section 4, we present and analyze the experimental results. Finally, we summarize this paper in Section 5.

Atmospheric scattering model
To describe the formation of hazy images, McCartney [5] proposed an atmospheric scattering model in 1976. Later, Narasimhan and Nayar [26,27] further derived the model, which describes the degradation process of the hazy images and is widely used in the dehazing of hazy images. Mathematically, the atmospheric scattering model can be described as: where I(x) is a hazy image, J(x) is a clear image, A is the atmospheric light, and t(x) is the transmission. The process of dehazing is actually that of calculating A and t(x) from hazy image I(x) to restore J(x) through the atmospheric scattering model. The transmission t(x) can be expressed as: where β is the atmospheric scattering coefficient, which is related to the wavelength of visible light, d(x) is the depth of the scene, which is the distance between the imaging device and the scene.

Dark channel prior dehazing algorithm
He [8] obtained a law by statistical analysis of a large number of outdoor haze-free images, that is, in most non-sky local regions, at least one color channel has some pixels whose intensities are very low and close to zero. This phenomenon is defined as the dark channel prior. These low pixel values are produced by shadows, colored objects, or the surfaces of darker objects. For the outdoor hazefree image J(x), the mathematical expression of the dark channel is: where J dark is the dark channel image, which is always low in theory and infinitely close to zero. J c is a color channel of J, and Ω(x) is a local patch centered on x. It can be clearly seen from Equation (3) that calculating the dark channel of a pixel is actually the process of finding the minimum twice.
It is known from the atmospheric scattering model that the transmission t(x) and the ambient atmospheric light value, A, of the image must be known before recovering the haze-free image. First, the dark channel map of the hazy image can be solved by the dark channel theory: Both sides of the above formula are simultaneously divided by A as follows: Assuming that atmospheric light A is a known value, the image transmission t(x) can be estimated by Equations (3) and (5): He [8] mentioned in the paper that even on sunny days, there are always some particles in the air. Generally, if you look at distant objects with the naked eye, you will still feel that the scenery is blurred. People can feel the depth of field in the scene through the haze. To be consistent with reality, a constant factor ω (0 < ω < 1) can be introduced in Equation (6) to retain some of the haze particles in the distance: The smaller the ω value is, the less obvious the dehazing effect will be. According to experience, ω generally takes a value of 0.95.
The dark channel prior assumes that the transmission characteristics of the light in the local area are consistent, but this assumption is often not satisfied in the edge regions where the depth is inconsistent due to the discontinuity of the scene depth. For a hazy image, when the depth of the object in the local area Ω(x) is the same, the dark channel image can be accurately obtained. However, when the depth of the field difference at the edge of the object in the local region Ω(x) is large, the transmission value estimation in the local region is not accurate. Therefore, it is necessary to refine the transmission map so that the haze-free image can be better transitioned at the edge of the scene without the white halos. At the same time, to improve the efficiency of the algorithm operation, this paper uses the guided filtering algorithm proposed by He [15], instead of the soft matting algorithm [9], to optimize the transmission map.
When both A and t(x) have estimates, the atmospheric scattering model can be used to obtain a haze-free image. Since the transmission rate may be too small in some areas to cause the image J(x) value to exceed 255, it is necessary to set a threshold t0 for t. In the study of He [8], t0 was taken as 0.1. When the obtained t is smaller than the threshold t0, the threshold t0 is taken. The final haze-free image recovery formula is:

Defect of DCP
Through a large number of experimental results, it is known that for some images containing large bright areas such as the sky and water surfaces, the dehazing results based on the dark channel prior will show significant color distortion in the bright areas. In fact, for outdoor clear images, the bright areas have a large pixel value, and channels with pixel values close to zero cannot be found in the area, so the dark channel priors will invalid in these areas [28]. As shown in Figure 1, the sky area and some bright white areas of the scene in the dark channel image have large pixel values.
(a) haze-free image (b) dark channel image of (a) If the dark channel prior is not considered, the transmission derived from Equation (5) is For bright areas that do not satisfy the dark channel prior, the value of the dark channel cannot be approximated to 0, so the denominator of the above formula is less than 1. From this, it can be inferred that the actual transmission tactual(x) of the bright regions is greater than the transmission t(x) estimated based on the dark channel prior. Therefore, the transmission map estimated based on the DCP algorithm is limited to the bright region, and the pixel channel of the sky region is divided by the relatively small t(x) (as shown in Equation (6)), which significantly increases the small difference between the pixel channels of the sky area. This causes the color of the restored image to be distorted.

Tolerance mechanism to correct transmission and its defects
As described in Sec. 2.3, the dark channel prior is ineffective in bright areas. This causes the transmission rate to be underestimated and results in color distortion in the restored image. If you want to eliminate color distortion, you must adjust the transmission of the bright area so that the estimated t(x) more closely matches the actual transmission tactual(x). To solve this problem, a tolerance coefficient K is introduced to determine the difference between the haze image I(x) and the atmospheric light factor A. If the absolute value of the difference is less than K, the area is a bright area. The corresponding transmittance of this area needs to be recalculated. If the absolute value of the difference is greater than or equal to K, the area is a non-bright area and the transmission of this area is kept constant. Formulated as follows: The recovery formula for a haze-free image is: Equation (10) is actually a supplement and improvement to the dark channel prior. The tolerance mechanism is introduced to ensure that the calculated transmission of the bright region does not have a large deviation from the actual transmission. After the simulation, for most images with sky regions, the dehazing effect after introducing the tolerance mechanism is better than the algorithm using only the dark channel prior. Many pieces of literature have done related experiments and found that most of the haze images can obtain better dehazing effects when the tolerance K is 50. However, it is found that there are two defects in introducing tolerance mechanism after analyzing Equation (10). a). The value of the tolerance K directly affects the correction of the transmittance and the dehazing effect. The fixed tolerance value does not effectively correct the transmittance of different image features. As shown in Figure 2, the first image has the best dehazing effect when K=80, and the sky region appears distorted when K=20. In the second image, the branches above the center lost considerable detail when K=80, and the image details were the best when K=45. Observe the area at the end of the road in the third image. The dehazing effect is best when K=20. b). If we only use the ratio of the fixed tolerance K to |I(x)-A| to determine whether the pixel point I(x) belongs to the bright region, so as to determine whether tolerance is introduced in the restoration of the pixel. It is easy to misjudge whether the pixel belongs to the bright region, and the dehazing effect becomes worse due to the wrong adjustment of the transmission. Figure 3(a) shows an image with a dense fog but no large-area sky area. Figure 3(b) shows the recovery result after the processing of Figure 3(a) by the He's algorithm without tolerance. Figure 3(c) is a restored image obtained by processing Figure 3(a) after using the fixed tolerance value 50 to correct the transmittance based on the He algorithm. It can be seen from Figure 3(c) that the tolerance mechanism is used to make the region lose a lot of image details.

Calculation of adaptive tolerance
It can be seen from Figure 2 that selecting a suitable value of tolerance can make the bright area of the restored image undistorted and retain more effective information. Therefore, we need to calculate an adaptive value of tolerance to recover the hazy images with different characteristics. After many experiments, we find that the larger the proportion of bright areas in the whole image, the higher the K value required to recalculate the transmission. When the bright area is small in the image (for example, yellow clouds or branches and poles in the sky will reduce the proportion of the sky in the whole picture), taking the smaller K value will retain more details. At this point, the problem is converted to how to calculate the adaptive K value by the proportion of the bright area so that it can have a good recovery effect for all images. Similar to introducing the tolerance K, we introduce |I(x)-A| to record the difference between pixel I(x) in the hazy image and the atmospheric light A.
The average image contrast reflects the concentration of the haze to some extent. In addition, through experiments, we further found that images with different contrasts have different dehazing effects on images with different tolerance values. The average contrast of the images is positively correlated with the tolerance value. The larger the average contrast of the image, the larger the tolerance value. Therefore, we use the average contrast as the threshold parameter for calculating the tolerance, and use it as an index and compare it with |I(x)-A| to determine whether the pixel belongs to a bright area. The formula is as follows: The experimental results show that when α is set to 2, there is a better dehazing effect. In the above formula, Inum is the number of pixels in the entire image, and the contrast ɑ*C is a threshold, which is used to determine whether the pixel belongs to the bright region. If the value of |I(x)-A| is less than the threshold, it means that the pixel belongs to a bright area, and the numerator represents the number of pixels belonging to the bright area in the entire picture. C is the contrast of the image, and the calculation formula is as follows [29]: where intensities Iij are the i-th j-th element of the two-dimensional image of size M by N. Imean is the average intensity of all pixel values in the image. Figure 4 shows the transmission map and restored images obtained using different algorithms. The first line is the dehazing effect after using the algorithms of He [8], and its transmittance image is obtained directly by using Equation (7) after using the DCP algorithm. So the transmittance of the distant sky area is close to zero, and it can be clearly seen that the color distortion of the sky area is serious. The second line is the dehazing effect after introducing the empirical tolerance to correct the transmission rate. It has some improvement compared with the algorithm of He [8]. The third line is the transmission map and the restored image obtained by using the improved algorithm of this paper. The adaptive tolerance K=62 is calculated according to the characteristics of the image itself, as shown in Equation (12). The transmittance value of the distant sky region of the second and third rows of Figure 4 is calculated by the Equation (10) to be between 0 and 1. It can be seen from Figure 4 that the improvement effect of the adaptive tolerance determined by the image feature on the color distortion of the sky region is most obvious.

Threshold of tolerance mechanism
We know that the value of adaptive tolerance K needs to be used to adjust the transmission only when the dark channel prior is invalid in the sky area or in the bright area of the scene, while other regions use the original dark channel prior to calculating the transmission, it can also obtain a better dehazing effect. Unfortunately, when there is a dense haze area or a third-party direct light source (sunlight, headlights, reflected glare, etc.) in the image, it is misjudged as sky area because the brightness of these areas is similar to the brightness of the atmosphere. As a result, the transmission of these regions is excessively increased, resulting in a serious loss of image detail in the region.
We find that the common feature of the bright areas of misjudgment is that the proportion in the image is small, basically no more than 5% of the entire image's pixel. Therefore, we must first determine whether there is a large area of bright areas in the image before calculating the value of tolerance K. If the bright area does not exceed 5% of the entire image's pixels, indicating that there is no large bright area in this image, then the value of tolerance should be as small as possible to maintain the original transmission for the dehazing calculation. If the bright area exceeds 5% of the entire image's pixels, then there is a large area of the sky in this image. At this time, it is necessary to calculate the value of adaptive tolerance K using Equation (12) to adjust the transmission. The final formula of tolerance K is as follows: where the numerator in the formula is the number of pixels that satisfy the condition, and Inum is the number of pixels in the entire image. We set the threshold Ith to 190 because the pixel values of the bright regions are relatively average and high. The 5% mentioned in Equation (14) is a threshold value to decide whether or not the image introduces adaptive tolerance to modify the transmission function. This parameter was determined after extensive experimental observation. As shown in Figure 5, the proportion of the bright areas of the five images is 3.223%, 1.78%, 5.78%, 6.761%, and 27%, respectively. The first three images do not have a large area of the sky, and the proportion of bright areas is less than 5%. The details of the image can be saved without using a tolerance mechanism. The last three areas have sky areas, and the bright areas account for more than 5%. Using the tolerance mechanism can reduce the distortion of the sky area image. Figure 5 shows the restored image corresponding to the different judgment values of whether or not to use the tolerance mechanism in this paper. Setting the threshold to 5% to determine whether to introduce adaptive tolerance to modify the transmission function can enhance the effectiveness of the proposed algorithm for different pictures.

Comparison and analysis of experiential results
He [8] calculated the first 0.1% of the pixel points from the dark channel image of the hazy image according to the brightness and then calculated the value of atmospheric light A by looking for the pixel-point of the corresponding position in the original image. However, if the picture has a thirdparty direct light source such as sunlight, headlights, and reflected strong light. The value of the atmospheric light obtained by He [8] is obviously too large, which will affect the final dehazing effect.
To overcome these limitations, this paper refers to the quad-tree subdivision method [30] to estimate atmospheric light A. First, the input image is divided into four matrix regions, and the average pixel value of each block is calculated. Then, the block with the largest average is selected and further divided into four smaller matrix regions. This process is repeated until the size of the selected area is less than a prespecified threshold (typically the threshold is set to 5%*w*h, where w and h represent the width and height of the input image). The defogging algorithm in this paper is the result of the quad-tree subdivision method and the adaptive tolerance correction transmittance.
The transmittance map and the restored image obtained by processing the haze image of Figure 4 by the improved algorithm of the paper are as shown in Figure 6. To verify the effectiveness of the proposed algorithm, we test various hazy images and compare them with Tarel [7], Cai [13], He [8], and the algorithm of introducing fixed tolerance values. The comparison includes a qualitative, subjective evaluation and quantitative, objective evaluation. Figure 7 shows the recovery effect of the improved algorithm and the other four representative dehazing algorithms on outdoor blurred images. The transmission map during the various dehazing algorithms is shown in the first row of Figure 7. Figure 7(a) depicts the hazy images to be dehazed. Figure 7(b-e) shows the results of Tarel [7], Cai [13], He [8] and the result of the introduced fixed tolerance algorithm. As shown in Figure 7(b), most of the haze is removed in Tarel's results, and the details of the scenes and objects are well restored. However, the result is clearly affected by over enhancement, making the entire image much darker than what it should be. In contrast, the results of He's image are much better visually (see Figure 7(c)). However, in the first two pictures, the sky area still shows significant distortion. This is because the method of He's transmission rate estimation is based on the dark channel prior, and the accuracy of the estimation largely depends on the validity of the dark channel prior. Unfortunately, the dark channel prior is ineffective when the scene brightness is similar to the atmospheric light. As shown in Figure 7(d)(e), the distortion of the sky region is improved. However, as shown in the sixth line and its enlarged image, the restored images of the two algorithms have lost detail in the region of dense haze. Compared with the results of these four algorithms, our results are not over-saturated and better preserve the details of the image. As shown in Figure 7(f), the sky area in the picture is clear, and the details of the mountains and leaves are well preserved.

Qualitative Comparison of Synthetic Images
In Figure 8, the five algorithms, including the proposed algorithm, are tested on the stereo images where the ground truth images are known. Figure 8(a) shows the hazy images, which are synthesized from the haze-free images with known depth maps. The results of the five algorithms are shown in Figure 8(b-f). Figure 8(g) shows the ground truth images for comparison. These hazefree images and their corresponding ground truth depth maps are taken from the Middlebury stereo datasets [31,32]. It is obvious that Tarel's results are quite different from the ground truth images, as the results are much darker (as shown in Figure 8(b)). By observing the images in Figure 8(c), we find that He's results have a similar problem (for example, in the third image, the skin color of the child is obviously deepened, and the color of the background behind the hat in the second image is darkened). Figure 8(d) is the recovery result of Cai et al. [13], and Figure 8(e)(f) is the recovery result of the fixed tolerance value algorithm and the adaptive tolerance algorithm in this paper. They maintain the original color of the object, and the result is that there is no oversaturation such that the corrected image is more similar to the ground truth images.

Quantitative Comparison
The subjective evaluation uses the naked eye to observe the contrast picture before and after dehazing to judge the effect of it. This method is often susceptible to individual factors, such as the aesthetics and psychology of the observer, and cannot be imported into the computer vision system to achieve more accurate and detailed follow-up analysis. Objective evaluation is more automatic, efficient, and easier to integrate than a subjective evaluation. Therefore, we use objective evaluation methods to further evaluate the various algorithms described in this paper.

Blind contrast enhancement assessment
In studying the dehazing effect, Hautière's blind evaluation method based on the visible edge contrast is well known [33,34]. This method evaluates the contrast enhancement of each image detail before and after dehazing. It uses three indicators to objectively describe the quality of the images  (18) where n0 and nr are the numbers of visible edges of the image before and after the dehazing, respectively, Ψr is the set of visible edges for the dehazed image, Pi is the pixels on the visible edge, ri is the ratio of the Sobel gradient at Pi and the corresponding point of the original image, ns is the number of saturated black and white pixels, dimx and dimy represent the width and height of the image, respectively. The comparative data are shown in Table 1. Table 1. Indicator e, r and σ of the images in Figure 6.  Table 1 shows the results of the three objective indicators e, r and σ of the image in Figure 6. A lower σ value represents better performance. Table 1 shows that the σ value of the algorithm in this paper is the smallest. Even if the σ value of the fifth picture is higher than that of He's method, the difference is small. The two indicators e and r focus on edge recovery and usually larger values indicate better performance. However, the increase in the visible edge may also result from a false edge caused by severe color distortion (as shown in Figure 6(b)). Figure 9 shows it more concretely, where Tarel's result shows evident false edges in the local enlarged regions. Our method suppresses the color distortion effectively, which is further manifested as a decrease in false fake edges. Thus, our results often have a lower value than the results of Tarel and in those two indicators. However, our method is actually better than most other methods in most images (as shown in Table 1). The structural similarity (SSIM) image quality assessment index [35] is introduced to evaluate the ability to preserve the structural information of the algorithms. This indicator was first proposed by the Image and Video Engineering Laboratory at the University of Texas at Austin. It is often used in image processing, especially in image denoising, which comprehensively surpasses SNR and PSNR in image similarity evaluation. The high SSIM indicates that the haze-free image is highly similar to the ground truth image, while the low SSIM indicates the opposite.

Methods
To directly compare the structural similarity between the restored image and the real image, we compare the dehazing results of the synthesized depth image in Figure 7. Table 2 shows the SSIM of the four restored images in Figure 7. The SSIM results of Tarel are all less than 0.8, indicating that a large amount of structural information is lost in the restored image. It is clear that Cai's SSIMs are higher than those of the other four algorithms. Cai's algorithm is a deep learning algorithm that continuously trains to obtain the best-restored image by learning the relevant features of the synthesized depth image. The algorithm can save structural information well. Our results reach the highest SSIM results except for Cai. It is proved that the ability of the algorithm to save structural information is superior to the general algorithm. All the simulations for our proposed algorithm are carried out in MATLAB R2016a environment running on a personal computer with an Intel Core (i3-8100) central processing unit running at 3.6 GHz with 4 GB of RAM. The average computing time for dehazing processing of our algorithm is less than 0.5 seconds. We think it is possible to apply it to a video including several frames in the future.

Conclusion
The dark channel prior is invalid when the scene brightness is similar to that of atmospheric light, and there is no shadow projection. Therefore, this paper improves the dehazing algorithm based on a dark channel prior. First, we estimate the atmospheric light using the method of quadtree subdivision and then combine the dark channel prior to introduce a value of adaptive tolerance K to obtain the correct transmission. To prevent the occurrence of halo artifacts at the edges of the restored image, the guided filtering technique is used to optimize the transmission. Finally, these parameters are brought into the atmospheric scattering imaging model to complete the dehazing treatment. The experimental results show that our dehazing algorithm not only better recovers images with large areas of bright areas but also better preserves image details compared with other methods. In future work, we will extend our work to the issue of video dehazing.