Haze Removal Based on Refined Transmission Map for Aerial Image Matching

: A novel strategy is proposed to address block artifacts in a conventional dark channel prior (DCP). The DCP was used to estimate the transmission map based on patch-based processing, which also results in image blurring. To enhance a degraded image, the proposed single-image dehazing technique restores a blurred image with a refined DCP based on a hidden Markov random field. Therefore, the proposed algorithm estimates a refined transmission map that can reduce the block artifacts and improve the image clarity without explicit guided filters. Experiments were performed on the remote-sensing images. The results confirm that the proposed algorithm is superior to the conventional approaches to image haze removal. Moreover, the proposed algorithm is suitable for image matching based on local feature extraction. Author Contributions: Conceptualization, Y.R.M. and O.-S.K.; methodology, Y.R.M. and O.-S.K.; investigation, Y.R.M. and O.-S.K.; writing—original draft preparation, Y.R.M. and O.-S.K.; writ-ing—review and editing, Y.R.M. and O.-S.K.; supervision, O.-S.K.; project administration, O.-S.K.


Introduction
Image matching is inevitably challenging when different images of a large scene are being matched. Although local feature points are generally used to match the same location in different images, image matching is difficult in terms of viewpoint, illumination, and temporal changes. There are two types of image-matching method: area-based and feature-based methods [1]. Area-based methods are used to assess the similarity of the pixels between the two images. In feature-based methods, salient features are directly extracted rather than intensity values. In early remote-sensing systems, this work was performed manually. However, this process can now be performed automatically using of local feature point detectors and descriptors [2,3].
However, in the case of outdoor scenes, image degradation [14] also occurs because of atmospheric conditions, such as haze [15][16][17][18], smoke, snow [19], fog [20,21], and rain [22][23][24]. Haze is particularly problematic for remote-sensing applications [25] when the images are captured from above the earth's atmosphere. Haze is caused by absorption, scattering, and re-emission in the atmosphere. Therefore, in the case of bad weather, images taken with a camera are degraded by haze depending on the distance. As a result, the degraded images lose their contrast and color fidelity.
Conventional methods for removing haze use additional hardware [26][27][28][29][30][31], multiple images [32], or additional information [33]. Multiple-image-based dehazing methods remove haze through two or more input images. These methods were developed by Tan [34], in which haze is removed by maximizing the local contrast based on statistics. However, because this approach is not a physical model, it only provides an improvement. Tarel and Hautiere [35] proposed a fast visibility restoration method to preserve edges and corners based on a median filter; however, this method is not suitable for images with a discontinuous depth. Ancuti et al. [36] recently proposed a simple contrast-based dehazing method without any geometrical information or user interaction. Minseo et al. [37] used principal component analysis to enhance haze images by estimating accurate transmission with a random forest. Dan et al. [38] implemented an efficient quad decomposition algorithm to estimate the atmospheric light to solve the color distortion and background noise problems. This method was applied to synthetic and hazy datasets.
Fattal [39] assumed that the surface shading and transmission functions were locally and statistically uncorrelated to estimate the color of the haze. Moreover, He et al. [40] proposed the DCP method, in which it is assumed that dehazed images have local patches that contain a small number of pixels with low intensity in at least one color channel. However, this method produces block artifacts that are patch based. He et al. [41] also proposed a guided filter-based dehazing method, but this method only focused on edgepreserving filtering. Geet et al. [42] investigated a new color channel method to remove atmospheric scattering, and obtained channel values and atmospheric light. However, this method focuses on enhancing the quality of the dehazed image using a dual transmission map. Chang et al. [43] proposed a method to solve the underestimation of the transmission of brighter areas using a weighted residual map. Yutaro et al. [44] also focused on local patches to address structural detail losses using the normalized pixel-wise DCP. Image dehazing algorithms can be classified as shown by the table in Figure 1. In this article, a novel algorithm is introduced that can remove haze from a degraded single image. The proposed single-image-based algorithm restores a hazy image by refining a transmission map using an approach based on a hidden Markov model [45,46]. The proposed strategy enhances the image contrast by reducing the block artifacts from the patch-based image dehazing method. As a second contribution, the proposed de-hazing method is suitable for image matching based on local feature-embedded colors using the C-SIFT operator. Finally, image matching was evaluated using aerial-haze images, the NH-Haze dataset, and a manually annotated dataset. The remainder of this article is organized as follows. In Section 2, a refined transmission map using the proposed method and its implementation are described. In Section 3, the experimental results are presented and discussed. Section 4 concludes the work. Additionally, the assessment of the dehazed images was evaluated and compared using the PIQE [47].

Physical-Based Image Dehazing and Matching
Haze is a challenging weather phenomenon for remote-sensing aerial images because, in dehazing, scene transmission depends on unknown depth information. The physical model removes haze based on direct transmission maps. To address block artifacts, a novel method is proposed to estimate a refined transmission map without explicit guided filters. The method consists of two parts. The first is calculation of the DCP, and estimation of the atmospheric light and DCP transmission map. The transmission map of DCP is then refined using the hidden Markov random field and expectation maximization (HMRF-EM) algorithm, which is explained in Sections 2.1 and 2.2. Second, image-matching is performed using the scale-invariant feature transform (SIFT) algorithm to find the valid key-point matches, which is explained in Section 2.3.

Dark Channel Prior
In general, the method of modeling the haze in one image is expressed by Equation (1), which defines the haze image, original image, atmospheric light, and transmission.
where I(x) is an image containing haze, and J(x) is the brightness value of the original image without haze. A is the atmospheric light and t(x) is the transmission, which shows the degree to which the unseen light reaches the camera. The DCP [40] is based on the observation in dehazed outdoor images of a few pixels whose intensity values in at least one color channel are very low in non-sky patches, namely, red (R), green, (G), or blue (B); that is, the patch is composed of very low values with minimum intensity and the dark channel is defined in Equation (2).
where C J is the color channel of J, and is a local patch centered at x . Based on observation, with the exception of the sky region, the intensity of dark J is low and tends to be zero if J is a haze-free outdoor image. Thus, dark J is the dark channel of J, and the above statistical observation is the DCP. To remove haze in the image, the top 0.1% of the brightest pixels were obtained for the estimated atmospheric light in the dark channel, and the chosen pixels were haze-opaque. In the next step, the coarse estimated transmission is as follows: where c A  is the color channel of estimated atmospheric light, and  is chosen as (0<  <1) [40] to prevent the full elimination of haze because it causes the haze to result in an unnatural image with the loss of depth perception. We fixed the value of  to 0.95 for all subsequent experimental results in this study. The estimated transmission map t is refined by applying a soft-matting method. Finally, the estimated scene radiance is expressed as: where t0 is the lower limit of the refined medium transmission and A  is the estimated atmospheric light. When applying the steps of the DCP method, the haze removal results can be obtained. Algorithm 1 provides the procedure for each step, as follows, to estimate the scene radiance J  .

Correction of Transmission Map Using Hidden Markov Random Field
The application of an HMRF-EM algorithm [45,46,48] to estimate the transmission map is proposed. The HMRF-EM algorithm improves the transmission map and enhances the dehazing performance based on a double stochastic process that uses the maximum likelihood to remove and smooth the areas of block artifacts. The input image is represented by where P(x) is the prior probability that represents the Gibbs distribution and the probability of joint maximum-likelihood is expressed as follows: Here, is the Gaussian distribution of the variable Next, two assumptions are required that are used to approximate the proposed HMRF-EM algorithm. The prior probability is defined as: where U( ) x denotes the prior energy function. The second assumption is as follows: From these two assumptions, the variable (0) θ is initialized. The probability distribu- θ ) is also estimated. The labels are then estimated using (0) θ . During this process, the estimation of the maximum a posteriori (MAP) probability is utilized, as shown below.
Next, the posterior distributions of all  l L and i y are calculated as follows: and Next, the variables are updated using the     t i P l y as follows: Equations (5) The likelihood energy is defined in Equation (17) as follows: The prior energy is defined using U(x), as shown in Equation (18).
The clique potential is defined as in (19) for the neighboring pixels.
From the above equations, we obtain Equation (20) to smooth the edges of the block artifacts, as follows: where z is the assumption of the binary edge map (if 1  i z , the edge is not deteriorated; otherwise, the edge is deteriorated) and c V is the clique potential. Algorithm 1 explains each step to obtain a refined transmission map using the HMRF-EM method.

Scale Invariant Feature Transform with Embedded Color
The SIFT [49] operator is broadly used in image matching due to its robustness to the rotation of the image, scaling, and variations in luminance. This algorithm consists of the following five steps: scale-space extrema detection, key-point localization, orientation assignment, key-point descriptor, and key-point matching. Moreover, SIFT with embedded color (C-SIFT) [50] was used in this study. The C-SIFT algorithm is robust because the local features include color information for image dehazing. In general, the camera response, as shown in (17) (22) here,

λ)S (λ)dλ ,(i = R,G, B)
Although haze cannot be entirely removed by dehazing processing, it is assumed that xy i ζ is not a dominant component. Here, if it is assumed that the camera sensitivities are delta functions, Equation (21) can be revised for any wavelength k. Note that realistic camera sensitivities are not delta functions, which is an issue that is discussed later.
The color response at the Planckian locus of an illuminant for any pixel is defined as follows: Here, . The natural logarithm is used to separate the two components, namely, the surface and illuminant, using Equation (24) as follows: Following the determination of the color responses for each RGB channel, the ratio for each channel is calculated using the same process as follows: The simple transform is used to identify the two significant clues using Equation (26).
Equation (27) shows that the relationship between the coordinates of the log-chromaticity and the altered level in the illuminant is linear, whereas k R is not dependent on the illuminant color. In addition, the relationship between the log-chromaticity and change in the surface reflectance is also linear, as shown in Equation (29).
When using a real camera, because it is impossible to implement the abovementioned sensitivities using a delta function, the proposed method uses the channel filter function of Worldview-2 to consider the characteristics of aerial images, rather than the mathematical approach presented in Equations (22)- (29). Next, the transformation of a real camera response into a new response is explained. A linear regression model [52] is used.

Results and Discussion
The experiments were performed on four datasets to evaluate the performance of the proposed algorithm, namely, synthetic haze images, natural aerial images from Google, NH-HAZE [53,54], and manually annotated Datasets. NH-HAZE is a synthetic non-homogeneous haze dataset consisting of 55 outdoor images. The manually annotated dataset was created in the lab and consists of 18 images for testing the homogeneous aerial images. We present the results in Figures 2-8. The experimental results show that the proposed method performs well on all the datasets.    Figure 3 shows the test images and the results obtained using the proposed method with aerial images. To evaluate the proposed method in Figures 3-5, natural aerial images were obtained from Google. The image sizes were 3264 × 2448, 4318 × 3083, 950 × 691, and 3008 × 2000. The performance of the proposed algorithm was also compared with those of several state-of-the-art dehazing methods when using aerial images, as shown in Figure  4. The proposed algorithm was used to process and analyze several large aerial images with a resolution of 3000 × 3000 pixels. The Figure 4 are the input hazy images are compared with the proposed method and conventional methods such as Tarel et al. [35], and He et al. [40], [41]. The proposed un-optimized (MATLAB implementation) code could process each aerial image in approximately half an hour using a desktop computer (Intel 5, 3.1-GHz CPU, 8-GB RAM). The proposed method produced visually pleasing results that were superior to those produced by conventional methods. The results obtained using the proposed algorithm were generally sharper and clearer than those obtained using the method of He et al.  Comparative test results of different dehazing methods using manually annotated dataset with Tarel and Hautiere [35], He et al. [40], He et al. [41], and the proposed method. artifacts can occur in the case of mixed clouds and haze (e.g., very bright regions) when using block-based processing. However, the proposed algorithm reduces the number of block artifacts, as shown in Figure 5d.The reduction of block artifacts is not clear in the aerial images of the proposed method. Therefore, we compared their transmission maps, as shown in Figure 6. We can clearly see how the block artifacts have been smoothed compared to the DCP. For example, Figure 6a is a DCP transmission map containing block artifacts, and Figure 6b is a refined transmission map using the proposed method with reduced block artifacts. However, some of these may be affected by uncertainties; therefore, they require fuzzy preprocessing [55][56][57]. Figures 7 and 8 show the results for non-homogeneous and homogeneous images. Also, the Figures 7 and 8 are the input hazy images are compared with the proposed method and conventional methods such as Tarel et al. [35], and He et al. [40], [41]. The proposed approach and the conventional methods, as listed in Table 1, were compared with the help of the PIQE [47] metric to evaluate the quality of the de-hazed image. The PIQE metric is used for human perceptual quality, which is inversely correlated with perceptual quality. Thus, a low score indicates good perceptual quality, and a high score indicates poor perceptual quality. The bold text indicates the best performance and the darkgreen text indicates the next-best performance. For instance, the low value for the first image in Figure 4 is that of the proposed method, with 15.5900, and the highest value is that of He et al. [41], with 34.5783. The same procedure was followed for the other tested images. Of all values of the metric, the proposed method demonstrates the best performance. The bold text indicates the best performance and the dark-green text indicates the next-best performance. Hence, the results suggest that the proposed method yields better results on tested datasets and aerial images. Finally, the effect of enhancing the image contrast by eliminating the haze in an image is emphasized by the image-matching results shown in Figure 9. For the C-SIFT pointmatching results, an experiment using a significant number of test images was conducted to support the point-matching related claims. When the C-SIFT operator was applied to the original hazy images, only 32 valid matches were produced. When the C-SIFT operator was applied to images enhanced using the method of Tarel and Hautiere [35], 31 valid matches were obtained. When the C-SIFT operator was applied to images enhanced using the method of He et al. [40], 74 valid matches resulted. Finally, when the proposed method based on C-SIFT was used, 85 valid matches were generated. Additionally, we show the valid key point matching techniques compared to different methods in Table 2. Of these methods, the proposed technique demonstrates more valid matches due to the reduction in block artifacts.

S. No
Tested Image Tarel

Conclusions
A dehazing method for image contrast enhancement was proposed. The method modifies the transmission map based on the DCP using the HMRF-EM algorithm. The method can enhance the image contrast by reducing the block artifacts caused by a patchbased image dehazing algorithm. The performance of the proposed algorithm was found to be superior to that of conventional methods for synthetic, outdoor, and aerial images. The algorithm is suitable for image-matching based on local feature points and outperforms conventional methods. The algorithm has certain limitations. For example, it can only be applied to RGB-based images, excluding satellite images without a blue band. The performance of the algorithm is also degraded when haze affects near-infrared channels. The performance of the delta functions, one of the basic assumptions of the proposed algorithm, can also be adversely affected when realistic camera sensitivities are used.