A Modified GrabCut Using a Clustering Technique to Reduce Image Noise

Abstract

In this paper, a modified GrabCut algorithm is proposed using a clustering technique to reduce image noise. GrabCut is an image segmentation method based on GraphCut starting with a user-specified bounding box around the object to be segmented. In the modified version, the original image is filtered using the median filter to reduce noise and then the quantized image using K-means algorithm is used for the normal GrabCut method for object segmentation. This new process showed that it improved the object segmentation performance a lot and the extract segmentation result compared to the standard method.

1. Introduction

Digital image processing deals with a wide variety of applications ranging from biology, military, medical, space science, art, games and movie industries.

The object segmentation is an important step in image processing and analysis [1]. In computer vision, segmentation divides the input image into background and objects. The purpose of the segmentation is to simplify and make it easy to interpret or convert to more meaningful representation of an image. Segmentation is one of the most difficult subjects in an digital image processing, and many studies on this subject have been done to get more accurate results.

GrabCut method is based on object segmentation algorithm called GraphCut [2,3]. While GraphCut algorithm segments an image without user intervention, GrabCut accepts an interest area defined by a user and extracts objects using the clues given to get better results. Many studies have been done to improve performance of GrabCut detecting objects in unknown regions [4,5].

In the proposed method, the image is smoothed using median filter and the quantized using k-means clustering technique. Then, GrabCut extracts objects from the quantized image [6]. In this way, we got improved performance.

2. Related Work

In general, object segmentation is one of the most fundamental tasks in image processing. Image segmentation is to divide an image into a number of pixel sets on the basis of shape or area. In this work, we following image filters and a clustering technique is applied for an efficient object segmentation.

2.1. Image Filter

Filtering is one of the main tasks of signal processing. Filtering is used to remove noise in the image, to extract visual characteristics of interest, and to resample the image. Representative filters are Gaussian filter, Mean, etc.

Gaussian filter is used to remove the noise using the following equations:

$$G(x)=\frac{1}{\mathsf{\sigma }\sqrt{2\mathsf{\pi }}}\exp \left[\frac{-x^2}{2\mathsf{\sigma }}\right]$$

(1)

$$G(x,y)=\frac{1}{2\mathsf{\pi }{\mathsf{\sigma }}^2}\exp \left[\frac{-(x^2+y^2)}{2{\mathsf{\sigma }}^2}\right]$$

(2)

where $\mathsf{\sigma }$ is the standard deviation.

Mean filter is the representative noise removing filter and is defined as follows:

$$\begin{array}{l}O(x,y)=\frac{1}{M\times M}{\displaystyle \sum _{i,j}^{M}I\left(x+i,y+j\right)}\\ M\in W\\ M=2N+1\end{array}$$

(3)

where $I(x+i,y+j)$ is a neighbor pixel and W is a mask of size $\left(2N+1\right)\times \left(2N+1\right)$.

2.2. Image Segmentation

Segmentation is the process of dividing the digital image into a set of multiple pixels to simplify the image representation. Typical methods of image clustering are Mean Shift (MS), Fuzzy C-Means (FCM), etc. [7].

Mean shift (MS) is a procedure for locating the maxima of a given window area by selecting a pixel (mode) most close to the averaged color, then moving the center of the window to the mode to find local maxima repeatedly until it is converged [8,9]. The result of MS is good in low frequency areas, but it has some difficulties to group high frequency areas. The following Figure 1 show the example.

Figure 1. MS clustering. (a) Original image; (b) Result image.

FCM is to overcome the difficulty of MS by considering belonging or membership degree into the distance. Data-points close to a cluster center have a high belonging degree [10,11].

Let $X=x_1,\cdots ,x_N\subseteq R^p$ a data set to be clustered, where p is feature dimension, $R^p$ is a real vector space, and N is the number of pixels. Each pixel of color image is expressed as feature vectors like $x_k=x_{k1},\cdots ,x_{kp}$ and center of cluster is $V=\left(v_1,\cdots ,v_C\right)$, where p is feature dimension and C is the number of clusters. FCM calculates matrix U minimizing the target function $J_{FCM}(U,V|X)$.

$$J_{FCM}(U,V|X)={\displaystyle \sum _{i=1}^C{{\displaystyle \sum _{k=1}^N\left(u_{ik}\right)}}^m{\Vert x_k-v_i\Vert }^2}$$

(4)

where ${\displaystyle \sum _{i=1}^C{u}_{ik}=1}$, and $u_{ik}$ is membership degree that data k belongs to cluster i. $V_j$ is the center of cluster i, defined using Equation (5).

$$v_i=\frac{1}{{\displaystyle \sum _{k=1}^N{u}_{ik}}}{{\displaystyle \sum _{k=1}^N\left(u_{ik}\right)}}^m{x}_k,m⟩1$$

(5)

where $u_{ik}$ is the mean value of $x_k$ using fuzzy constant m. One disadvantage of FCM is that the number of clusters should be provided in advance. Figure 2 is the class membership map applying FCM algorithm and Figure 3 is result of FCM clustering.

Figure 2. FCM class membership map. (a) Class 1; (b) Class 2; (c) Class 3.

Figure 3. FCM clustering. (a) Original image; (b) Result image.

Image segmentation can be accomplished also by using GraphCut. Boykov proposed GraphCut to get optimized interactive image segmentation:

$$E\left(\mathsf{\alpha }\right)=U\left(\mathsf{\alpha }\right)+V\left(\mathsf{\alpha }\right)$$

(6)

where $\mathsf{\alpha }$ is a vector of either 0 or 1. 0 means background and 1 means object. $U(\mathsf{\alpha })$ is continuity between adjacent pixels and $V(\mathsf{\alpha })$ is a data term representing how much the data belongs to object or background. Data term requires prior information about object and background, and provided probability density function, data term calculated using Equation (7).

$$V\left(\mathsf{\alpha }\right)=\{\begin{array}{c}{\displaystyle \sum _{p\in P}-\log p\left(I_p|H_b\right)if{\mathsf{\alpha }}_P=0}\\ {\displaystyle \sum _{p\in P}-\log p(I_p|H_o)\mathrm{otherwise}}\end{array}$$

(7)

where $H_b$ and $H_o$ are histograms of background and object, p is a pixel and P is a pixel set of an image. $I_p$ is the intensity of pixel P. Figure 4 is result of GraphCut algorithm that is another typical method of image segmentation.

Figure 4. GraphCut algorithm. (a) Original image; (b) Result image.

3. The proposed Method

The flow of proposed method is show in Figure 5.

Figure 5. A flowchart of the proposed method.

3.1. Clustering to Reduce Noise

Digital image can be contaminated during data transmission.

$${\displaystyle \sum _{i=1}^N\left|Xmed-\mathrm{{\rm E}}\right|}\le {\displaystyle \sum _{i=1}^N\left|Y-Xi\right|}$$

(8)

where N is the size of data set. K-means clustering method is applied to the output of the filter. K-means classifies the data set to the predefined number of classes. Let ${\mathsf{\mu }}_i$ be the center of i-th cluster and $S_i$ be the set of pixels belongs to cluster i. The variance of all the data set is defined as Equation (9).

$$V={\displaystyle \sum _{i=1}^k}{\displaystyle \sum _{j\in S_i}}{\left|x_j-{\mathsf{\mu }}_i\right|}^2$$

(9)

The goal is to find $S_i$ minimizing V. K-means starts with arbitrary initial values ${\mathsf{\mu }}_i$. Allocating pixels to close ${\mathsf{\mu }}_i$ and recalculating ${\mathsf{\mu }}_i$ is repeated until it is converged.

$$\begin{array}{l}{J}_{MSE}={\displaystyle \sum _{i=1}^K}{\displaystyle \sum _{x\approx {\mathsf{\omega }}_i}}{\left|x-{\mathsf{\mu }}_i\right|}^2\\ {\text{where\hspace{1em}}\mathsf{\mu }}_i=\frac{1}{n}{\displaystyle \sum _{x\approx {\mathsf{\omega }}_i}x}\end{array}$$

(10)

Equation (10) is the simplest clustering method minimizing $J_{MSE}$ repeatedly. Figure 6 shows applied the median filter in image.

Figure 6. An image applied the median filter image. (a) Original image; (b) Result image.

3.2. Object Segmentation Using Improved GrabCut

GrabCut accepts an interest area defined by a user and extracts objects using the clue given. Figure 7 shows trimap of foreground of GrabCut algorithm.

Figure 7. Composition of trimap; shows trimap of foreground ($T_F$), background ($T_B$) and unknown region ($T_U$).

Object and background is mixed in unknown region. Background $T_B$ is defined as Equation (11).

$$T_B=T_F\cap \left[\left(T_U\cup \mathrm{\Gamma }\left(z\right)\right)\oplus S\right]$$

(11)

where $\mathrm{\Gamma }\left(z\right)=\left\{z_n|{\displaystyle \sum _{p\in N\left(z_i\right)}\nabla g\left(p\right)⟩t}\right\}$ is the area greater than gradient. Symbol $\oplus$ is a dilation operator and S is a structure element for it.

A Gaussian mixture model is a probabilistic model that assumes all the data points are generated from a mixture of a finite number of Gaussian distributions with unknown parameters and is given by:

$$p\left(x|\mathsf{\theta }\right)={\displaystyle \sum _{i=1}^{N}p\left(x|{\mathsf{\omega }}_i,{\mathsf{\theta }}_i\right)}P\left({\mathsf{\omega }}_i\right)$$

(12)

where i-th vector component is characterized by normal distributions with weights ${\mathsf{\alpha }}_i$ and a pair of mean and covariance ${\mathsf{\theta }}_i$. ${\mathsf{\omega }}_i$ represents relative importance. ${\mathsf{\alpha }}_i$ is defined as follows:

$$0\le {\mathsf{\alpha }}_i\le 1\text{\hspace{1em}and\hspace{1em}}{\displaystyle \sum _{i=1}^M{\mathsf{\alpha }}_i}=1$$

(13)

Parameters for GMM (Gaussian Mixture Model) of M components is expressed as Equation (14):

$$\begin{array}{l}\mathsf{\theta }=({\mathsf{\mu }}_1,{\mathsf{\mu }}_2,\dots ,{\mathsf{\mu }}_M,\dots ,{\mathsf{\theta }}_1^2,{\mathsf{\theta }}_2^2,\\ \dots ,{\mathsf{\theta }}_M^2,{\mathsf{\alpha }}_1,{\mathsf{\alpha }}_2,\dots ,{\mathsf{\alpha }}_M)\end{array}$$

(14)

$T_U$ is defined as follows:

$$T_U=T_F-T_B$$

(15)

Figure 8 shows a result of the improved GrabCut algorithm.

Figure 8. Improved GrabCut algorithm. (a) Original image; (b) Result image.

4. Experiment and Discussion

The experiments were performed using about 400 photos such as figures, plants, food, etc.

Performance of GraphCut, standard GrabCut and proposed method is compared. Figure 9 shows some results. GrabCut is better than GraphCut and, the proposed method shows better results than GrabCut in most cases by detecting background in unknown area [12,13].

Figure 9. (a) Original image; (b) GraphCut Algorithm; (c) GrabCut Algorithm; (d) Ref. [12]; (e) Ref. [13]; (f) Proposed Method.

Evaluation is performed using precision and recall. Precision is the fraction of retrieved instances that are relevant, while recall (also known as sensitivity) is the fraction of relevant instances that are retrieved:

$$precision=\frac{N\left(Ob{j}_{EX}\cap Ob{j}_{GT}\right)}{N\left(Ob{j}_{EX}\right)}$$

(16)

$$recall=\frac{N\left(Ob{j}_{EX}\cap Ob{j}_{GT}\right)}{N\left(Ob{j}_{GT}\right)}$$

(17)

where $N\left(·\right)$ is the number of pixels, $Ob{j}_{EX}$ is the object and $Ob{j}_{GT}$ is ground truth objects. Figure 10 shows the precision and recall of three methods. The proposed method gives the best result.

Figure 10. Precision-recall result.

Experiments are performed using PSNR (Peak Signal to Noise Ratio). PSNR is the ratio between the maximum possible power of a signal and the power of corrupting noise that affects the fidelity of its representation.

$$\begin{array}{l}PSNR=10\cdot {\log }_{10}\left(\frac{MA{X}^2}{MSE}\right)\\ =20\cdot {\log }_{10}\left(\frac{MA{X}_I}{\sqrt{MSE}}\right)\end{array}$$

(18)

MSE (Mean Squared Error) is the difference between the estimator and what is estimated. Where MSE is defined as follows:

$$MSE=\frac{1}{mn}{\displaystyle \sum _{i=0}^{m-1}}{\displaystyle \sum _{j=0}^{n-1}}{\Vert I(i,j)-K(i,j)\Vert }^2$$

(19)

Table 1 shows the quantitative comparison of result experiments.

Table 1. Result of experiment.as quantitative comparison.

Comparision Method	Ref. [2]	Ref. [13]	Proposed Method
MSE	2748.65	2374.43	1308.17
PSNR	29.39 dB	34.17 dB	38.94 dB

5. Conclusions

In this paper, a modified GrabCut method is proposed using median filter and k-means clustering technique to reduce image noise and to extract objects better. An image is preprocessed and then used for the input of standard GrabCut. This method showed better performance than GraphCut or standard GrabCut from the various and complex pictures like medical images, traffic images and people images. This research should be extended further to detect objects in video, and this can be used in many industrial applications.