Salt and Pepper Noise Removal Method Based on a Detail-Aware Filter

: The median-type filter is an effective technique to remove salt and pepper (SAP) noise; however, such a mechanism cannot always effectively remove noise and preserve details due to the local diversity singularity and local non-stationarity. In this paper, a two-step SAP removal method was proposed based on the analysis of the median-type filter errors. In the first step, a median-type filter was used to process the image corrupted by SAP noise. Then, in the second step, a nov-el-designed adaptive nonlocal bilateral filter is used to weaken the error of the median-type filter. By building histograms of median-type filter errors, we found that the error almost obeys Gaussian–Laplacian mixture distribution statistically. Following this, an improved bilateral filter was proposed to utilize the nonlocal feature and bilateral filter to weaken the median-type filter errors. In the proposed filter, (1) the nonlocal strategy is introduced to improve the bilateral filter, and the intensity similarity is measured between image patches instead pixels; (2) a novel norm based on half-quadratic estimation is used to measure the image patch- spatial proximity and intensity similarity, instead of fixed L 1 and L 2 norms; (3) besides, the scale parameters, which were used to control the behavior of the half-quadratic norm, were updated based on the local image feature. Experimental results showed that the proposed method performed better compared with the state-of-the-art methods.


Introduction
Detail restoration is a challenging problem that is necessary for some image processing tasks, such as feature extraction, object identification, and pattern recognition. The impulse noise remains in the process of image acquisition and transmission [1,2], which is an inevitable and unwanted phenomenon. The salt and pepper (SAP) noise is a type of impulse noise where the corrupted pixel takes either the maximum or minimum gray value. This noise appears as white and black pixels in the corrupted image [3][4][5]. Moreover, the challenge in detail restoration is further amplified when the images are corrupted by heavy SAP noise owing to the significant destruction of information detail.
In this work, a two-step SAP noise removal method is proposed. The aim of this method is to study the denoising result of the median-type filter further, and propose a novel method to improve the visual quality. First, by analyzing the error of the median-type filter, it was observed to adhere to a Gaussian-Laplacian mixture distribution statistically. Following this observation, in the second step, an adaptive nonlocal bilateral filter has been proposed to utilize the nonlocal feature and bilateral filter to recover the result of median-type filters. In the proposed filter, the difference between the image patch is measured by a modified version of the adaptive norm proposed in [6], which is different from the traditional methods. Moreover, the scale parameters, which were used to control the behavior of the adaptive norm, were updated based on the local feature for higher estimation accuracy.
The contributions of this paper are summarized as follows: (1) By analyzing the error of the median-type filter statistically, it is found that the error is almost a Gaussian-Laplacian mixture distribution. Therefore, a two-step noise removal method is designed to remove the SAP noise. (2) A novel adaptive non-local bilateral filter is proposed to recover the median-type filtered result. Owing to the drawbacks of traditional bilateral filter, a nonlocal operator is used to extract image patches, and the adaptive norm is used to measure the spatial proximity and intensity similarity between the patches. (3) We propose a method to calculate the scale parameters in the adaptive norm. Using this strategy, the context information can be utilized to make the norm adapt to the patch feature.

Related Work
There are several methods proposed to reconstruct images corrupted by SAP noise in the past. The research for nonlinear filter is actively pursued in the field of SAP noise removal. Median-type filters [7][8][9][10] are the most popular nonlinear filters. Such as the standard median filter (MF), which is one of the most popular nonlinear filters. MF convolves a moving window with determined size over the image. If the pixel at the center of the window is noisy, its value is replaced with the current window's median value. However, the noise-free pixels are treated similar to the noisy pixels when these filters are used to restore the corrupted image. These pixels are also replaced with estimated values, which lead to artifacts and blurring. This results in (1) the distortion of original image details and (2) the increase in the computation, especially in the case of low signal-noise ratio.

Switching Filters
To solve this problem, the switching median-type filters are proposed. The main idea of switching filters is to use a switching process to select the optimal output in the noise detection step or correction step. In [11], first, the authors propose impulse noise detection methods based on evidential reasoning. Then, they design an adaptive switching median filtering which could adaptively determine the size of filtering window according to detection results. The quaternion switching vector median filter, introduced in [12], detects impulse noise based on a quaternion-based color distance and local reachability density. The quaternion-based color distance is used to calculate the local density of a color pixel. Chanu and Singh [13] also propose a switching vector median filter based on quaternion to remove impulse noise, which contains two stages. In the first stage, they use a rank strategy to determine whether the central pixel of the filtering window is noisy pixel or not. Then, in the second stage, the probably corrupted candidate is re-confirmed by using four Laplacian convolution kernels. The noisy pixel is processed by a switching vector median filter based on the quaternion distance.

Decision Filters
The decision-based method has attracted significant attention for removing SAP noise. Decision filters assume that noisy pixels have a value of 0 or 255 while noise-free pixels have a value between them. As presented in [14], a new filter is established on decision-based filters. The method contains noise detection and restoration. In restoration phase, the uncorrupted pixels keep unchanged and corrupted pixels are interpolated from surrounding uncorrupted pixels using a two-dimensional scattered data interpolation, named natural neighbor Galerkin method. In [15], the noise-free pixels are also left unprocessed and the noisy pixels are replaced with the Kriging interpolation value. The noisy pixels are interpolated from the noise-free pixels in a confined neighborhood. The weights of contributors are calculated using semi-variance between the corrupted pixels and uncorrupted pixels. Besides, an adaptive window size is used for the increasing noise densities.

Fuzzy Filters
Toh and Isa [16] propose the noise adaptive fuzzy switching median filter (NAFSMF), which is a hybrid between the simple adaptive median filter [17] and the fuzzy switching median filter [18]. NAFSMF contains two stages: noise detection and removal. In the detection stage, the histogram of the corrupted image is utilized to identify noisy pixels. In the removal stage, fuzzy reasoning is employed to deal with uncertainty and design a correction term to estimate noisy pixels. In [19], the authors first improve the maximum absolute luminance difference (ALD) method for detecting noisy pixels more accurately, which helps to classify the pixels into three categories: uncorrupted ones, lightly corrupted ones, and heavily corrupted ones. To restore corrupted pixels, a distance relevant adaptive fuzzy switching weighted mean filter is implemented to remove noise. Afterward, the uncertainty present in the extracted local information as introduced by noise is handled via fuzzy reasoning [20,21].

Morphological Filters
Morphological filters are non-linear and it can modify the geometrical features locally [22,23]. Mediated morphological filters, introduced in [23], are a combination of median filtering and classical gray-scale morphological operators. After, the mediated morphological filters are applied to remove noise from adult and fetal electrocardiogram (ECG) signal [24,25] and medical images [26]. In [24], mediated morphological filters are used as an efficient preprocessing step to deal with adult and fetal ECG signals. Such a strategy could suppress noise effectively and show low sensitivity to the changes of the structuring element's length. Compared with [24], the method presented in [25] further concludes the preprocessing result by employing a morphological background normalization. In [26], the mediated morphological filter ability is verified in removing SAP noise, Gaussian noise and Speckle noise from medical images. Compared with the weighted MF, classical MF, and linear filter, the mediated morphological filter shows better performance, even in high abnormal condition of each kind of noise.

Cascade Filters
The cascade methods use the idea of combining different filters to improve the restoration quality. Such as, the authors of [27] combine the switching adaptive median with fixed weighted mean filter (SAMFWMF). This filter contains switching adaptive median filtering and fixed weighted mean filtering with additional shrinkage window. This filter can achieve optimal edge detection and preservation. In [28], a decision-based median filter is combined with asymmetric trimmed mean filter. In this method, the pixel whose value is equal to 0 or 255 is replaced with the median of the moving window; otherwise, it is replaced with the mean of the window. Raza and Sawant [29] combine a decision-based median filter with a modified decision-based partially trimmed global mean filter (DBPTGMF). Esakkirajan et al. [30] combine a decision-based median filter with a modified decision-based unsymmetrical trimmed median filter (MDBUTMF). Both DBPTGMF and MDBUTMF contain two stages. The first stage is common for both approaches: detect the noisy pixels and replace them with the window's median. Two main differences exist in the second stage: (I) when detecting the corrupted pixels in the window, if all the pixels in the window (except for the central corrupted pixel) are 0 s (or 1 s), DBPTGMF replaces the corrupted pixel with 0 (or 1). On the other hand, MDBUTMF replaces the corrupted pixel with the window's mean. (II) If the pixels in the window are a combination of 0 s and 1 s, DBPTGMF replaces the corrupted pixel with the mean of the window. However, the corrupted pixel will be replaced with the median of the window if at least one pixel in the window is different from 0 or 1. For both cases, MDBUTMF replaces the corrupted pixel with the median of the window.

Nonlocal Means Filter and Bilateral Filter for SAP Noise Removal
Most of the existing SAP noise removal methods are pixel-based methods, which consider the image pixels in a fixed local region only and ignore the image self-similar information. Thus, the image nonlocal structure or the texture structure cannot be preserved properly. To solve this problem, Wang et al. [31] propose the iterative nonlocal means (INLM) filter to remove SAP noise. In this method, switching median filter is first used to mark the pixels as noisy or noise-free pixels and performs filtering on the noisy pixels only. Then, an iterative nonlocal means framework is used to estimate the noisy pixels. In [31], they iteratively exploited the nonlocal similarity feature of the image. They also obtain higher accuracy by updating the similarity weights and the estimated values simultaneously. The bilateral filter is a widely used method for Gaussian noise suppression, and it is barely discussed for SAP noise removal. Veerakumar et al. [32] propose an adaptive bilateral filter by modifying the spatial proximity-based and intensity similarity-based Gaussian function. In the intensity similarity-based Gaussian function, the difference between different pixels is measured by the L2 norm, which penalizes the high-frequency component, and this may blur the edge and texture.

Error Analysis
In this work, we focused on the statistical characteristics of the results other than the method itself. Based on this idea, we first add salt and pepper noise with various intensities to the image and then remove the noise using the median type filter (i.e., MF, NAFSM) where is the error image between the filtered image ˆm f I and the original image Iori.
Then, we compute the normalized histogram for the error image. Take the image "Lena" as an example, Figure 1a is the error image between I and the original image Iori, Figure 1b is the histogram for Figure 1a, Figure 1c is the error image between the filtered image and the original image Iori . . To verify our hypothesis, we repeated these experiments in different images with different sizes and noise intensities. Furthermore, the histogram of the error of these images is shown in Figure 1d. It can be seen that the curves corresponding to different images are all very close to the red curve (i.e., Gaussian distribution defined in Equation (3)) at both ends, while close to the black curve (i.e., Laplacian distribution defined in Equation (4)) near the peak area. It means that after prefiltering, the noise remained in the image obeys a Gaussian-Laplacian mixture distribution.
If r obeys Gaussian distribution, i.e., ~( , ) G G r μ σ Ν , the distribution function is defined as where, G μ and L μ are position parameter, G σ and L σ are scale parameters.
Intuitively, we deduce that the image obtained by the median-type filter is similar to an image with mixture noise, which can be written as This phenomenon gives us an inspiration that a better denoising performance would be achieved by successfully designing a Gaussian-Laplacian mixed noise filter and introducing it into a SAP noise removal problem. In this paper, this question will be discussed and satisfactory results are obtained by applying the modified bilateral filter into salt-and-pepper noise removal.

Bilateral Filter
Bilateral filter [33,34] is a kind of nonlinear filter aimed at edge preservation, since it simultaneously considers spatial proximity and the intensity similarity between image pixels. Mathematically, the corrupted image pixel can be estimated by the weighted average of all the neighborhood pixels as   (6) where x(i, j) denotes the estimated image pixel located at (i, j), B(i, j) denotes the patch whose center pixel is located at (i, j), ( , ) ( , ) I k l B i j ∈ , and are both Gaussian functions to measure the spatial proximity and the intensity similarity, respectively. Formally, these functions can be written as where, σd and σr are smoothing parameters.

Proposed Two-Step Algorithm
This section presents a detailed explanation of the proposed two-step SAP noise removal algorithm. In the first step, a median-type filter (i.e., NAFSMF) was used to process the corrupted image. Then, in the second step, a novel-designed adaptive nonlocal bilateral (ANB) filter is used to weaken the error of the median-type filter given that it is found to have a Gaussian-like distribution statistically.

NAFSMF for Preprocessing
In this stage, the noisy image histogram is utilized so as to estimate the noisy pixels in the image corrupted by SAP noise. The local maximum method [18] is first used to detect the noisy pixels. By using this method, it can avoid mistaking noise free intensities in the noisy image histogram for the noisy intensities in the case of low noise intensity. The local maximum is the first peak encountered when traversing the noisy image histogram in a particular direction. The search is started from both ends of the histogram and directed towards the center of the histogram. Two noise intensities are found and used to identify possible noisy pixels in the image. The two local maximums are denoted as NSalt and NPepper, respectively. A noise mask M is designed as follows to mark the location of noisy pixels where I(i, j) records the gray value at point To calculate the noisy point I(i, j), first, define a search window 2 1 ( , ) Then, is used to count the number of noise-free pixels in , there is not enough noise-free pixels in the current window. In this case, 1 s s Finally, y is used to denote the filtering result and the central pixel y(i, j) is calculated using the equation where F(i, j) is a fuzzy membership function that is defined as (14) In Equation (14), D(i, j) represents the local information defined as the maximum gray value difference in a 3 × 3 window, and it is defined as

ANB Filter to Improve Result
The bilateral filter has local characteristics due to considering the pertinence of pixels to estimate noisy pixels. The pertinence will be destroyed when encountering high noise intensity, thus the denoising effect will be greatly reduced. To address this problem, we first introduce the nonlocal strategy to the bilateral filter, and a new function to measure the patch intensity similarity is obtained, where, Ω represents the search window, ( , ) i j Θ represents the patch centered by y(i, j), ( , ) k l Θ represents the patch centered by y(k, l), 0 β > is a model parameter.
Suppose the vector n v R ∈ , the Lp norm of v is defined as the Gaussian noise and Laplacian noise, L2 norm is an excellent choice. However, the L2 norm is outliers, thus, the L2 norm does not perform as well as the L1 norm in preserving sharp edges. Moreover, when using Equations (7) and (16) based method to deal with the preprocessing result, there still exists some problems, such as, (1) all pixels need to be estimated; (2) all pixels in the search window contribute to the estimation; and (3) fixed L2 norm used in Equation (16) is sensitive to high frequency information, which may blur the image edge. Based on the above analysis, we further modified the spatial proximity-based function and the intensity similarity-based function as  (18) where, Ω represents the search window, Θ(i, j) represents the patch centered by y(i, j), Θ(k, l) represents the patch centered by y(k, l), 0 α > and 0 β > are model parameters, a) has self-adaptability of mimicking L1 and L2 norms, and the parameter a controls the transition from L1 norm to L2 norm. Larger a denotes a larger range of difference values that can be discriminated by the L2 norm. In the two patches, Θ(i, j) and Θ(k, l), the difference y(p, q) − y(r, s) may be variational owing to the location. Thus, using the same a to measure the difference is not an excellent choice. In this work, the parameter a is computed based on the local image feature (19) where ε > 0, which is used to avoid making the dividend zero. Detail analysis about , , , p q r s a is presented in Section 5. Thus, by substituting a with Equation (19) in Equation (18), we can obtain new functions for the distance in intensity space  (17) and (20), the ANB filter is proposed and defined as In the following section, we will discuss the advantages of the adaptive norm compared with L1 and L2 norms in theory. The experimental comparisons are also shown in Section 6.

Proposed Two-Stage Noise Removal Algorithm
In this subsection, we summarized the whole salt and pepper noise removal algorithm. The whole Algorithm 1 contains two stages: (1) median-type filter and (2) the proposed ANB filter. The proposed algorithm is described as

Efficiency Analysis
One of the main contributions of this manuscript is the establishment of the ANB filter, which is based on the improved functions ( and ˆr ω ) with regard to the spatial proximity and the intensity similarity, respectively. The efficiency analysis mainly includes three aspects: (1) the error analysis, which provides an explanation to add the following ANB step; (2) the weight analysis, which shows the efficiency of the introduction of and ˆr ω into the ANB filter; and (3) the norm choice analysis, which presents the efficiency of using the adaptive norm to improve ˆr ω . The error analysis has been presented detailed in Section 2, thus, in this section, the efficiency analysis mainly includestwo parts: weight analysis and norm choice.

Weight Analysis
Two improved weight functions were designed in the proposed filter. The two functions were based on spatial proximity and intensity similarity. The efficiency of using the two weights and ˆr ω was analyzed in three special situations as follows: Case 1: Considering Equations (17) and (20), if the contributed pixel y(k, l) is corrupted, and . To this end, the pixel y(k, l) has no effect on estimating y(i, j). This design can avoid misestimation owing to the corrupted points.
Case 2: If the contributed pixel y(k, l) is noise-free in the smooth region, but y(i, j) in an area where the gray value changes dramatically (i.e., edge region), the difference between the two pixels is large. Correspondingly, the difference between the patches centered by y(k, l) and y(i, j) may be large. In this case, the variable ˆr ω is close to 0, and the value of is small. Thus, it is a double insurance to keep the weight of y(k, l) is small enough when computing for y(i, j).
Case 3: If both y(k, l) and y(i, j) are in the smooth region, a small difference is shown among the image pixels. In this situation, the difference between the patches tends to be 0 infinitely. Thus, the weight ˆr ω approaches 1, and only the weight works. Here, the proposed filter works as a Gaussian filter.

Norm Choice
The proposed norm avoiding to set the threshold value to control the selection of the norm compared with other estimators (i.e., the Huber norm, the Leclerc norm, and the Lorentzian norm). From Equation (19), it is noted that when the difference between y(p, q) and y(r, s) is small, the value of ap,q,r,s is large. Here, the modified adaptive norm contains a range where it performs similar to the L2 norm as large as possible. Conversely, ap,q,r,s is small, and the modified adaptive norm performs similar to the L1 norm, which can remove noise and preserve edges. Thus, the proposed adaptive norm can work as L1 norm or L2 norm automatically according to the local features.
In the following equation, where ap,q,r,s is used to control the scope of the linear behavior as shown in Figure 2. This norm is shown in Figure 2 with different ap,q,r,s-values where it can be seen that the adaptive norm gets closer to the L1 norm when the sale parameter tends to zero. Specifically, larger sale parameter results in a larger range of error values that can be discriminated by the linear influence function. Moreover, in ( ) tion from L1 norm to L2 norm.

( )
, , , r i j k l ω , Equation (8), L2 norm was used to measure the difference between the point y(i, j) and y(k, l). Theoretically, when the difference is large, the weight of y(k, l) is correspondingly small, which means that y(k, l) has little impact on the estimation of y(i, j). Meanwhile, when the difference is very small, especially close to 0, y(k, l) is similar to y(i, j). Here, the L2 norm can enlarge the weight of y(k, l), which increases its contribution to y(i, j). Sometimes, the expected effect may not be achieved using the L2 norm. The intensity difference in the smooth region tending to be zero is an ideal condition; thus, using the L2 norm to control the weight based on the intensity similarity is not an optimal choice. A simple example is shown in Figure 3 where the region in the red block is an almost smooth area,which is noted as region A. Assuming that ( , ) , ( , ) y i j A y k l A ∈ ∈ , y(i, j) is a corrupted pixel, and y(k, l) is uncorrupted. Intuitively, y(k, l) should make a large contribution for estimating y(i, j), namely, ( , , , ) 1 i j k l ω → , where y(i, j) is the center value of region P1 and y(k, l) is the center value of region P2. However, in the real situation, this assumption could not be achieved because the L2 norm will sharpen the difference. However, the adaptive norm would have a good performance. The analysis is discussed as follows. . Specifically, using the L2 norm to measure the difference will weaken the contribution of y(k, l), which is contrary to the proposed norm.

Experimental Results and Discussion
To evaluate the proposed algorithm, several simulation experiments were performed in this section to compare with many state-of-the-art methods. The noisy images are generated by adding salt and pepper noise with nine different intensities from σ = 10% to σ = 90% to the test images shown in Figure 4, which presents gray-scaled image with sizes between 256 × 256 and 512 × 512. Denoising performance was measured by many subjective and objective standards. The subjective standard included reconstructed images and the corresponding error images. The objective standard included image enhancement factor (IEF) [35], structure similarity index (SSIM) [36], and peak signal-to-noise ratio (PSNR) to evaluate the performance of the proposed method.
In the proposed method, the related parameters are set as follows: T1 = 10, T2 = 30, α = 50× noise intensity, β = 50× noise intensity, the similar patch size is 3 × 3, and the searching window size is 7 × 7. For the other methods, the parameters are set for the best performance. All codes were running in MATLAB 2015b using an i7-7500U CPU, 16 GB RAM computer under Microsoft Windows 10 operation system. For each comparison method, all parameters were selected under the declarations in their papers.

Comparison under Different Norm Choice
We will take Barbara and Man-made images shown in Figure 4 as examples to compare our proposed method with the L1-based method and L2-based method. The main difference between these three methods is the norm used to measure the intensity similarity in the second stage. Tables 1-3 show the PSNR value, SSIM value, and IEF value, respectively. According to the results in Tables 1-3, the proposed method outperformed the other two methods in most cases, especially in high-intensity noise. For example, the proposed method performs better than the L1-based method (average PSNR(dB): 0.68, 0.66, and 0.12; average SSIM: 0.0151, 0.0178, and 0.0099; average IEF: 10.93, 5.85, and 1.09) in the case of =70% σ , =80% σ , and =90% σ , respectively. Moreover, the proposed method performs better than the L2-based method (average PSNR(dB): 0.69, 1.1, and 1.1; average SSIM: 0.0029, 0.0132, and 0.0172; average IEF: 6.65, 13.74, and 13.14) in the case of =70% σ , =80% σ , and =90% σ , respectively. The L1-based method has the advantage in edge preserving. However, the detail region would be over-smooth. The L2-based method can remove noise effectively, while it always makes the edges blur. The proposed method is based on an adaptive norm which has self-adaptability of mimicking L1 and L2 norms. Thus, our method can balance the noise removal and detail protection, even in high-intensity noise.

Comparisons between Pre-and Post-Processed Images
In this sub-section, we would like to show the proposed method effectively reducing the error for a preprocessed image. Figure 5 shows three examples based on Couple, Pepper, and Street images. For better comparisons, we show the error images and the enlarged detail parts. From Figure 5, we can see that the post-processed images have more acceptable details and less error. Besides, the corresponding PSNR, SSIM, and IEF values are presented in Tables 4-6. According to these tales, it can be seen that the post-processed images achieves higher PSNR, SSIM, and IEF values. For example, the post-processed images gains higher values than the pre-processed images ( , respectively. This indicates that the proposed method could further reduce the error in the preprocessed images. That is because in the proposed two-stage method a novel bilateral filter is used to weaken the median-type filter errors is effective. Figure 5. Comparisons of the pre-and post-processed images. In each line, the left part is the original image, the middle part is the zoomed details. The pre-processed marked "1", the post-processed marked "2". The right part is the corresponding error images, the first one is pre-processed error, the second one is post-processed error.

Subjective Quality Analysis
In this sub-section, three evaluation measures (i.e., PSNR, IEF, and SSIM) were presented to validate the perception-based image quality assessment on different images under different noise levels. We compared the proposed adaptive method with DBA, MF, ACWMF, NASFM, NASEPF, INLM, DAMF, and FSAP. For far comparison, all parameters in each algorithm are under their statements in published papers.
The graphs of IEF for different images with noise levels varying from σ = 10-90% are shown in Figure 6. For the subfigures in Figure 6, x-axis means σ = 10-90% from left to right. The y-axis denotes the IEF value. This figure confirmed that the proposed method obtained better results compared with other considered methods in terms of higher IEF value. Table 7 shows the PSNR values on the restored images corresponding to eight test images, including those shown in Figure 4 with the noise intensity changes from 10% to 90% as well as Table 8 shows the corresponding SSIM values on the restored images. For better comparisons, we have colored the best values into red.
From these tables, we can see that our proposed method gets the highest PSNR and SSIM values both in the low noise intensity and high noise intensity, which demonstrates the efficiency and necessity of incorporating a novel bilateral filter to weaken the median-type filter errors. In the case of low noise intensity, taking σ = 10% as an example, all the methods achieve high PSNR and SSIM values. We can see that in the case of σ = 10%, the PSNR value of the proposed method (38.15 dB) has outperformed that of the MF method, the ACWMF method, the DBA method, the DAMF method, and the FSAP method more than 1.2. Even in the case of σ = 90% (correspondingly a strong noise), the PSNR value of the proposed method (22.81 dB) has outperformed the PSNR values of the MF method, the ACWMF method, the DBA method, the DAMF method, and the FSAP method more than 1.4. In addition, the proposed method also achieves slightly better reconstruction performance than the INLM method (22.27 dB). Compared with the INLM, a novel bilateral filter is proposed in this work, which is more efficient to suppress the Gaussian-Laplacian errors.

Objective Quality Analysis
This section presents the visual results obtained by different methods. We evaluated the performance of the proposed method and the methods under comparison via reconstructed images and the corresponding error images, which showed that the error image presented the difference between the original image and the reconstructed image. In the error image, the white point was the error between the de-noised image and the original image. The whiter point denotes the larger difference.
The subjective analysis of the proposed method against the existing methods with different noise intensities were shown in Figures 7-10. In these figures, the proposed method was performed in different texture types of images under σ = 60%, 70%, 80%, and 90%.
To explore the visual quality of different methods, we showed the reconstructed images with different noise densities. Figures 7 and 8 show the restoration results of various methods for the test images, which were corrupted by salt and pepper noise with 60% and 70% noise density, respectively. The MF and ACWMF were also found to fail in restoring the corrupted image. In the reconstructions obtained by DBA and NAFSM, many artifacts exist, which make the contours of the estimated images indistinguishable. The DBA, NAFSM, INLM, DAMF, and FSAP can restore the image with better quality. However, the error noise still exists in the reconstructions. Therefore, the results indicated that the proposed method can preserve details better than other methods. Figures 9 and 10 show the restoration results of different methods for images that were corrupted by heavy salt and pepper noise with 80% and 90% noise density, respectively. In these figures, the first and third rows were the original images and reconstructed images, respectively. The second and fourth rows were the noised image and corresponding error images, respectively. The SM and DWM filters failed to restore images corrupted by the heavy noise. The visual perception of the reconstructions obtained by MDWM filter was bad because some obvious white and black pixels still exist in the restored images.      -10 show that the proposed method achieved better visual quality. Our proposed method can preserve texture regions or edge regions by adaptively calculating the nonlocal region features and estimate the original pixel value. Thereby, the proposed method reduces reconstruction error as far as possible. Moreover, in the existing methods, undesirable artificial artifacts were inevitably produced in the reconstructed images. As we expected, satisfactory visual effects obtained by our developed algorithm were more natural and have fewer artifacts, especially in high noise intensities.

Effect of Searching Window
In this subsection, we will explore the denoising effect of the proposed method under several searching windows with different size, including 7 × 7, 11 × 11, 15 × 15, and 21 × 21. We take four images as test images, in which two 256 × 256 (i.e., Dog and Zebra images) and two 512 × 512 images (i.e., boat and Man images), and nine noise levels from σ = 10-90% are used to generate synthetic noising images. Table 9 presents the average PSNR and SSIM values of the test images under the contribution of different searching windows. In addition to these two quality metrics, another important aspect is the testing speed. Figure 11 shows the average run times of the proposed method with different searching windows for denoising images with 10 noise levels.
From Figure 11 and Table 9, we can see that the methods based on searching windows with 7 × 7 and 11 × 11 have high speed. With the noise intensity increases, the speed is lower. Compare with the methods based on searching windows with 7 × 7, 11 × 11, 15 × 15, respectively, the method based on 21 × 21 window does not achieve obvious higher PSNR and SSIM values. Although more pixels contribute to the estimation, not all pixels are beneficial to restore the corrupted pixels. Moreover, its speed is fairly slow due to the use of more pixels than other windows. Take the 256 × 256 images for example, the average running time, PSNR and SSIM values achieved corresponding to σ = 10% and searching windows with 7 × 7, 11 × 11, 15 × 15, 21 × 21 respectively are shown in the following: method based on 7 × 7 window (10 s, 30 (684 s, 18.66 db, 0.5440). Therefore, the searching window with 11 × 11 gains the best performance among all the cases when all of these factors are considered together.

Conclusions
This paper presented an adaptive nonlocal bilateral filter for salt and pepper noise removal. First, the NAFSM filter was introduced to distinguish the noisy and noise-free pixels and then conduct preliminary filtering on the noisy pixels. Second, an adaptive norm with scale parameters calculated based on local feature was designed to measure the intensity difference between image patches. Finally, the nonlocal thought, the bilat-eral thought, and the adaptive norm were combined. Then an adaptive nonlocal bilateral filter was designed to suppress the Gaussian-Laplacian mixture noise and further improve the reconstruction quality.
In Section 6, we demonstrate the benefit of our proposed method on the salt and pepper noise removal problem. We first conduct experiments to verify the effectiveness of the adaptive norm adopted in the novel bilateral filter. We have observed that adaptive norm gives comparable or better results compared to the L1 norm and L2 norm. Then, we demonstrate that the proposed bilateral filter could effectively weaken the median-type filter errors. Especially for the high noisy intensity (σ = 90%), the proposed bilateral filter can obtain satisfactory results. We demonstrate the capability of our proposed method with some state-of-art methods with different noise intensities from σ = 10% to σ = 90%. Numerical results illustrated that the proposed method outperformed the state-of-art methods with better visual quality and higher quality metric values. It indicates the effectiveness of our method to use a two-step framework based on the novel-designed adaptive nonlocal bilateral (ANB) filter. Finally, in order to explore the denoising effect of the proposed method under different searching windows, we also compare the denoising results with different window sizes, including 7 × 7, 11 × 11, 15 × 15, and 21 × 21. Comprehensively considering the running time, PSNR and SSIM values, the searching window with 11 × 11 is proven to be a better choice.