Image Enhancement for Surveillance Video of Coal Mining Face Based on Single-Scale Retinex Algorithm Combined with Bilateral Filtering

Abstract

Surveillance videos of coal mining faces have close relation to the safety of coal miners and mining efficiency. However, surveillance videos are always disturbed by some severe conditions such as atomization, low illumination, glare, and so on. Therefore, this paper proposed a hybrid algorithm (SSR-BF) based on the integration of single-scale Retinex (SSR) and bilateral filtering (BF) to enhance the image quality of surveillance videos. BF was coupled with SSR to reduce the noises and perfect the edge information in the image. The schematic diagram and pseudocode of SSR-BF was designed, and the parameters were set rationally to ensure the enhancement effects through some simulations. Finally, some comparisons with other methods were carried out, and the simulation results demonstrated that the proposed algorithm was superior to others and could be applied to image enhancement for poormonochrome images, especially the surveillance video of a coal mining face.

1. Introduction

Nowadays, video surveillance systems have been widely applied in many fields to monitor emergency actions or hazardous situations [1,2,3]. In a coal mining face, the video surveillance system plays an important role in the process of coal production, which can restrict the safety and efficiency of coal mining. However, due to poor coal mining environments, the image quality of surveillance videos is sensitively affected by the external environment, including by atomization, low illumination, glare, and so on. If this phenomenon exists in the surveillance video, the image quality will decline sharply, which means that it cannot be used to accurately monitor hazardous states of machinery and some safety accidents will also happen [4,5,6,7]. Therefore, there is an urgent need to study an image enhancement method for acquiring ameliorative surveillance video with a perfect vision effect.

The Retinex algorithm, first proposed by Land [8], is widely used in image enhancement areas [9,10]. Alessandro Rizzi and his group have done much revolutionary research in the field of Retinex algorithms [11,12]. The fundamental of Retinex is to divide an image into two parts, the illumination image and the reflection image, and then to remove the illumination image to realize image enhancement [13,14]. Single-scale Retinex (SSR) and multi-scale Retinex (MSR) are two typical Retinex algorithms, and MSR is defined as a weighted sum of several SSRs [15]. In recent years, many researchers have focused on the Retinex algorithm carefully and proposed some improvement strategies. Xie et al. [14] used guided filter to improve SSR and got good results in enhancing the image quality and finger vein recognition accuracy. Xiao et al. [16] employed a fast mean filtering to improve the performance of SSR. Fu et al. [17] tried simultaneous estimation of illumination and reflectance in the linear domain to improve Retinex, and their method was proved to be better than other common methods. Jang [18] proposed a subband-decomposed method to improve Retinex and enhance the contrast. Lee et al. [19] proposed an adaptive to improve the weights of MSR for image contrast enhancement. Shukri et al. [20] used homomorphic filtering to estimate illumination image and minimize the intra-individual variations in motion-blurred iris images. Wang and Huang [21] used a compute unified device architecture (CUDA)-enabled parallel algorithm to speed up Retinex. Banic and Loncaric [22] proposed memory sprays to enable a fast Retinex-based local image enhancement. Grigoryan et al. [23] utilized Fourier transforms to speed up Retinex.

Image noise is a common problem in the image processing field. Currently, some researchers have proposed some methods to wipe out the image noise. Zhou et al. [24] proposed a robust destriping algorithm with a spatially adaptive unidirectional total variation model to remove the noise of a multi-detectors imaging system. Zhou et al. [25] proposed a robust hybrid unidirectional total variation model of undesired noise imaged from a degenerated push broom scanner. Cao et al. [26] presented an improved histogram matching algorithm for the removal of noise in optical remote sensing imagery. Zhuang and Wang [27] brought forward a novel nonuniformity correction algorithm based on speeded up robust features extraction to reduce excessive noise and ghost artifacts. Shen et al. [28] proposed a piece-wise approach to removing the nonlinear and irregular stripes in moderate resolution imaging spectroradiometer (MODIS) data. Bouali et al. [29] improve the detection of ocean fronts from whiskbroom scanner images by removing the impact of striping on the observed gradient field. Rogass et al. [30] proposed a new approach to reduce uncorrelated striping noise and applied in hyperspectralpushbroom acquisition. However, few studies have been involved in image de-noising during processing images with a SSR algorithm.

Bilateral filtering (BF), proposed by Tomasi and Manduchi [31] in 1998, is a nonlinear spatial filter and has unique de-noising characteristics. As a kind of effective image de-noising technique, BF has been widely used in many fields [32,33]. In this paper, a hybrid algorithm (SSR-BF) is presented based on the combination of SSR and BF to process the images of surveillance videos. The rest of this paper is organized as follows. The basic principle of SSR and the integrated SSR-BF is presented in Section 2. In Section 3, the proposed approach is proved efficient by some simulation examples, and the comparisons are demonstrated to verify the de-noising ability and the actual image enhancement effect of the proposed approach. Our conclusions are summarized in Section 4.

2. The Proposed Image Enhancement Algorithm

2.1. Basic Principle of Single-Scale Retinex and Discussion

Retinex was first proposed by Edwin Land [8], and it tries to assure the color constancy under different lighting conditions. Essentially, Retinex segments the source image L(x, y) into two parts, the illumination image I(x, y) and the reflection image R(x, y), as shown in Equation (1):

$$L(x,y)=I(x,y)\cdot R(x,y),$$

(1)

where x and y are the coordinate index of pixel in the image.

As seen from Equation (1), the source image can be expressed as the product of the illumination image and the reflection image. The illumination image can reflect the characteristics of the light source, and the reflection image includes the substantive characteristics of the source image. By removing the illumination image and reserving the reflection image, the source image can be enhanced.

Based on the above principle, Jobson et al. [34] proposed SSR algorithm, and they proved that a Gaussian function could effectively estimate the illumination image from the source image. The principle of SSR can be depicted as Figure 1, and the calculation formulas are shown as Equations (2) and (3):

$$\log [R(x,y)]=\log [L(x,y)]-\log [L(x,y)\otimes G(x,y)],$$

(2)

$$G(x,y)=\frac{1}{2\pi {\sigma }^2}\exp [-(x^2+y^2)/2{\sigma }^2],$$

(3)

where G(x, y) denotes the Gaussian function and σ is the scale parameter. $\otimes$ denotes the convolution operation. With the increase of σ, the dynamic compression capability is strengthened and the details can be enhanced, but the color and anamorphose extrudes. Otherwise, the dynamic compression capability weakens and the color fidelity performs better. In general, to keep the balance between image color and edge information, σ can be set between 50 and 100.

In practical applications of SSR, some imperfections are encountered and they can be described integrally with Figure 2. Figure 2a is a foggy image, which is taken as the original image, and the result processed by SSR is shown in Figure 2b. As shown in Figure 2b, some noise points are gathering in the processed image, and the image quality is relatively poor. In general, SSR possesses the ability to eliminate atomization and enhance the contrast, but it cannot remove the noises hidden in the image.

2.2. The Bilateral Filtering Algorithm

As a kind of nonlinear spatial filter, BF can achieve uniform spatial filtering and retain the edge information of an image, which is an effective image de-noising method. BF is essentially designed to compute the weighted mean of similar pixel points with continuous space, and it adds a pixel distance judge function on the basis of ordinary Gaussian filtering. That is to say, BF can determine the weight of nuclear filtering according to the spatial distance and pixel distance. Figure 3 shows the principle of the BF algorithm. After the product filtering of weights, the noise of the output image can be effectively smoothed and the edge information can be retained.

The BF can be described as Equation (4):

$$\tilde{A}(\mathit{p})=\frac{1}{k(\mathit{p})}{\displaystyle \sum _{\mathit{q}\in \mathsf{\Omega }}{g}_d(\mathit{q}-\mathit{p})g_r[A(\mathit{q})-A(\mathit{p})]A(\mathit{q})},$$

(4)

where k(p) is a normalized constant and can be calculated as:

$$k(\mathit{p})={\displaystyle \sum _{\mathit{q}\in \mathsf{\Omega }}{g}_d(\mathit{q}-\mathit{p})g_r[A(\mathit{q})-A(\mathit{p})]},$$

(5)

where g_d denotes the Gaussian weight function of spatial distance and can be calculated as:

$$g_d(\mathit{x})=\exp (-{\Vert \mathit{x}\Vert }_2^2/2{\sigma }_d^2),$$

(6)

where g_r denotes the weight function of pixel distance and can be calculated as:

$$g_r(\mathit{x})=\exp (-x^2/2{\sigma }_r^2)，$$

(7)

In addition, p and q are two dimensional vectors, which express two adjacent points in the original image. $A(\mathit{p})$ denotes the pixel value at point p in the original image, and $\tilde{A}(\mathit{p})$ denotes the corresponding pixel value in the filtered image. Ω denotes the adjacent area around q. σ_d and σ_r are the scale parameters of g_d(x) and g_r(x), which have a crucial influence on the performance of the bilateral filter. An improper selection of σ_d and σ_r may result in the loss of image detail, residual noise, and other problems. However, no perfect theory can be used as the basis for selecting parameters, and a trial-and-error method is normally adopted to determine σ_d and σ_r, which cannot acquire the optimal performance.

In the image processing field, guided image filtering is a kind of filtering technology that has been introduced in recent years. The most common similarity to the bilateral filtering is that guided image filtering also has the edge retention properties. The two methods are both used to processed the image in Figure 2b and the reuslts are shown as Figure 4. Compared Figure 2b with Figure 4a,b, the noise is obviously reduced and the edge information can be retained more clearly. To reflect the comparison results of Figure 4a,b, the pixel differences of two graphs are computed as Figure 4c with the pixel threshold 10. Through the above comparison analysis, bilateral filtering and guided image filtering have very similar filtering capabilities. Hence, we alternatively select the bilateral filtering to combine SSR algorithm in this paper.

2.3. The Hybrid Image Enhancement Algorithm

In order to wipe out the noise in the process of SSR, a hybrid image enhancement algorithm is presented through the integration of SSR and the BF algorithm, abbreviated as SSR-BF. The original image should be firstly processed by the BF algorithm to reduce the noise and enhance the edge information. The block diagram of hybrid algorithm is plotted as Figure 5.

The pseudocode of the SSR-BF algorithm (Algorithm 1) can be shown as follows:

Algorithm 1

ImageEhance (InputImg, σ, S1, σd, σr, S2, OutputImg)

For x = 1 to M do //

For y = 1 to N do //

illusionImg(x,y) = Gussianblur(InputImg, S1, x,y)//Perform Gaussian filtering

End

End

For x = 1 to M do //

For y = 1 to N do //

DenoiseImg(x,y) = Bilateralblur(InputImg, S2,x,y)

End

End

logReflectImg = log(DenoiseImg)-log(illusionImg)

OutputImg = Nomorlize(logReflectImg, 0, 255)

3. Simulation Examples

3.1. Preperations

In order to verify the actual application effect for the surveillance video of a coal mining face, four images were captured from the surveillance video as the original simulation images. The images were influenced by severe conditions of atomization, low illumination, glare, and so on. The image size was 349 pixels × 286 pixels, shown as Figure 6.

The simulations were performed in the following environment: CPU i5—2450 M, memory 4 GB, graphics AMD 6630 M, Microsoft Visual Studio 2012 + OpenCV 2.4.11, and the algorithms in the following partsall ran with a system language of C++.

3.2. Parameters Selection of Proposed Method

To ensure that the SSR-BF algorithm possesses better performance, the parameters should be selected reasonably. In SSR-BF, there are five parameters, including σ, Gaussian filtering template size S₁, σ_d, σ_r and BF template size S₂. σ is usually set between 50 and 100, and we set σ equal to 75 in this paper. According to the literature [35], σ_d has less effect on the performance of BF, and larger σ_d will acquire a better smoothing effect. In this paper, σ_d was set to 100. Other parameters were selected as follows.

3.2.1. Selection of Parameter S₁

In the process of filtering, template size is an important parameter. For a Gaussian filter, different S₁ can influence the Gaussian estimation effect of an illumination image. To choose the optimal S₁, we set a series of odd values to simplify the index of the filter. Taking Figure 6a as an example, the enhancement effect of SSR with different S₁ was shown as Figure 7.

Through the subjective evaluation of Figure 7, SSR could obtain better enhancement effects with a larger value of S₁. When S₁≥ 21, the enhancement effect was basically the same. However, larger S₁ would increase the computation time and reduce the efficiency of the algorithm. Hence, comprehensively consideringthe enhancement effect and processing efficiency, we set S₁ = 21.

3.2.2. Selection of Parameter S₂

In analogy to selecting parameter S₁, we also preset a series of odd values for parameter S₂. The de-noising effect of SSR-BF with different S₂ was shown in Figure 8.

Seen from Figure 8, SSR-BF could remove noise more efficiently with a larger value of S₂. When S₂ ≥ 7, the SSR-BF represented a better de-noising effect. However, larger S₂ would also reduce the efficiency of the algorithm. In order to objectively evaluate the de-noising performance of SSR-BF, two indexes of mean square error (MSE) and peak signal to noise ratio (PSNR) were introduced in this paper [36]. In general, the perception of the image with higher PSNR and lower MSEis better.

Assuming two m × n monochrome images I and K, I is a processed image and K is the reference image. i and j denote the pixel coordinates of I and K. MSE and PSNR can be calculated as:

$$\begin{array}{l}MSE=\frac{1}{mn}{\displaystyle \sum _{i=0}^{m-1}{\displaystyle \sum _{j=0}^{n-1}{(I(i,j)-K(i,j))}^2}},\\ PSNR=10\times {\log }_{10}(MA{X}_I{}^2/MSE),\end{array}$$

(8)

where MAX_I is the highest grayscale in image I.

From Figure 8, SSR-BF possessed the best de-noising ability with parameter S₂ = 51. Therefore, the processed result of S₂ = 51 was set as the reference image. The MSE and PSNR values were calculated and illustrated as Figure 9. The changes of MSE and PSNR began to slow when S₂ ≥ 21. Hence, we set S₂ = 21.

3.2.3. Selection of Parameter σ_r

σ_r is a key parameter of a bilateral filter for controlling the edges information. In order to select an appropriate σ_r, this paper attempted to perform many simulations. The processed results of Figure 6a based on SSR-BF with different σ_r were shown as Figure 10, and the local regions in red boxes were enlarged as in Figure 11.

From Figure 10 and Figure 11, the noise could not be reduced efficiently when σ_r < 5, while the edges were too fuzzy for σ_r > 10. Therefore, σ_r was set from 5 to 10, and the hybrid algorithm could remove noise more efficiently and keep edges more clearly. To further evaluate the enhancement effect of on image, a comprehensive evaluation index A composed of PSNR and the fuzzy degree of grayscale images (FD) were adopted [37]. FD can describe the ambiguity of image, and the smaller the value, the more fuzzy the image. The calculated formula of A was given as follows:

$$\begin{array}{l}A=PSNR+FD,\\ FD={\displaystyle \sum _{k>T}k{H}_k},\end{array}$$

(9)

where k was the grayscale, H_k was the proportion of pixels with grayscale k, and T was the threshold and was chosen to be near the mean gray level value. We set the threshold equal to the first integer greaterthan or equal to the mean gray level value of the first image in each series.

For σ_r $\in$ [5, 10), the processed image with σ_r = 10 was set as the reference image of PSNR. Then the values of A with different σ_r were plotted as Figure 12. When σ_r = 8.9, the comprehensive evaluation index A achieved the maximum, and the processed image based on SSR-BF presented the best de-noising effect and edge information. Hence, σ_r was set to 8.9.

3.3. Results Analysis

3.3.1. De-Noising Effect Analysis

In order to manifest the de-noising effect of SSR-BF, four images in Figure 6 were processed by SSR and SSR-BF algorithms. The processing results were shown in Figure 13. We could find that the images processed by SSR-BF possessed more balanced brightness, more edge information, less noise, and better subjective visual effects than SSR. Furthermore, pixel value standard deviation (PSD) was adopted to reflect the noises objectively. Smaller values indicated it had a better de-noising effect. The PSD values of images were listed in

Table 1. Comparison of pixel value standard deviation (PSD) between the proposed method and single-scale Retinex (SSR).

Methods	Image (a)	Image (b)	Image (c)	Image (d)
SSR	54.84	56.78	53.67	55.67
SSR-BF	54.13	56.17	53.11	55.07

. Seen from Figure 13 and Table 1, the proposed SSR-BF algorithm was superior to standard SSR in terms of de-noising.

3.3.2. Image Enhancement Effect Analysis

In order to further verify the image enhancement performance of SSR-BF, five other methods of homomorphic filtering (HF), histogram equalization (HE), standard SSR, improved SSR in [38] (SSR1), and improved SSR in [39] (SSR2) were used to process the images in Figure 6, and the comparison results are shown in Figure 14.

From Figure 14, the defects of uneven brightness and unsharp edges in darker areas existed in the four images processed using HF and HE methods. The brightness and sharpness of the images processed by SSR, SSR1, and SSR2 was improved to some extent, but many noises still existed. The proposed SSR-BF method could enhance image clarity and contrast, simultaneously reduce the noises in the image, and keep the edge information more clearly.

In order to reflect the comparison objectively, two indexes of contrast (Con) and comentropy (Com) were introduced this paper to evaluate the images in Figure 14, which could be calculated as:

$$\begin{array}{l}Con=\sqrt{\frac{1}{mn}{\displaystyle \sum _{i=1}^{mn}{(I_i-\frac{1}{mn}{\displaystyle \sum _{j=1}^{mn}{I}_j})}^2}}\\ Com=-{\displaystyle \sum _{k=1}^K{H}_k\log H_k}\end{array}$$

(10)

where K was the sum of gray scales in an image.

The comparative contrast and comentropy of images processed by different methods were plotted in Figure 15. In general cases, larger contrast and comentropy of images indicate better image quality. However, some images have the opposite result, and the poor visual effect of the image has larger contrast and comentropy. For instance, the images enhanced by HE had the largest contrast and comentropy, while the images were overexposed and had the worst quality, as seen from Figure 14. From a comprehensive analysis of Figure 14 and Figure 15, we could assert that HE had the worst image enhancement performance. Hence, in the following analysis, the histogram equalization was omitted. In Figure 15, the contrasts of images processed by the SSR-BF algorithm were 57.79, 59.22, 56.72, and 56.73, which were larger than the other four methods. In addition, the comentropy of the images processed by the SSR-BF algorithm was 5.34, 5.39, 5.31, and 5.09, which was also larger than the other four methods. The simulation results indicated that the image enhancement effect based on SSR-BF was better than others. The analysis results were consistent with the subjective visual effect in Figure 14, and the comparison results proved the superiority to other methods in the aspect of poormonochrome images.

In order to show the clarity on outperformance of SSR-BF and draw statistical significances, 25 other images were intercepted randomly from a surveillance video of a coal mining face. These images suffered the interference of atomization, low illumination, and glare to some extent and they were also processed by the five methods of HF, SSR, SSR1, SSR2, and SSR-BF. The change curves of contrast and comentropy were plotted as Figure 16.

As seen from Figure 16, the mean values of contrast (56.08) and comentropy (5.15) based on SSR-BF were larger than HF (52.17, 4.77), SSR (47.99, 4.82), SSR1 (51.29, 5.05), and SSR2 (52.22, 5.06). Furthermore, the fluctuations of the contrast and comentropy curves obtained through SSR-BF were smaller than those of the other four methods. The comparison results indicated that SSR-BF had a better performance in estimating the enhancement effect of poor monochrome images, which could provide a more precise assessment of image quality.

4. Conclusions

In this paper, we proposed a hybrid algorithm based on SSR and BF to enhance the image quality of a coal mining face. The schematic diagram and pseudocode of the SSR-BF algorithm were designed, and some key parameters were determined reasonably according to many simulations. At last, the de-noising effect and image enhancement effect of the proposed method was analyzed and compared with other methods. The simulation results indicated that SSR-BF could reduce the noises in the image and simultaneously keep the edge information more clearly. The image enhancement effect of SSR-BF was better than HF, HE, SSR, SSR1, and SSR2 in term of visual assessment, contrast, and comentropy.