An Efficient SAR Image Segmentation Framework Using Transformed Nonlocal Mean and Multi-Objective Clustering in Kernel Space

Abstract

Synthetic aperture radar (SAR) image segmentation usually involves two crucial issues: suitable speckle noise removing technique and effective image segmentation methodology. Here, an efficient SAR image segmentation method considering both of the two aspects is presented. As for the first issue, the famous nonlocal mean (NLM) filter is introduced in this study to suppress the multiplicative speckle noise in SAR image. Furthermore, to achieve a higher denoising accuracy, the local neighboring pixels in the searching window are projected into a lower dimensional subspace by principal component analysis (PCA). Thus, the nonlocal mean filter is implemented in the subspace. Afterwards, a multi-objective clustering algorithm is proposed using the principals of artificial immune system (AIS) and kernel-induced distance measures. The multi-objective clustering has been shown to discover the data distribution with different characteristics and the kernel methods can improve its robustness to noise and outliers. Experiments demonstrate that the proposed method is able to partition the SAR image robustly and accurately than the conventional approaches.

1. Introduction

Synthetic Aperture Radar (SAR) is an advanced microwave equipment of earth observation and has obtained much more interests for it working at all-weather and all-time, strong permeability, multi-bands and polarization information [1]. Unfortunately, the images obtained by SAR are generally degenerated by speckle noise because of its distinct and special imaging mechanism. The existence of speckle usually leads to the degradation of image quality and has a directly impact on the SAR image recognition and interpretation. Furthermore, another important issue in SAR image understanding is how to efficiently distinguish different land covers in this type of images. That is to say, the fast and correctly partition or recognition of massive overlapped pixels into different local regions. In generally, the above two issues are two main crucial points in SAR image segmentation. Consequently, an efficient SAR image segmentation methodology should consider these two issues simultaneously.

The theory of fuzzy set is very suitable to SAR image segmentation because the pixel is the returning signal of radar waves reflected from many elementary scatterers within a resolution cell. Nevertheless, the resolution cell usually is denoted by an area of the land space, which may has regions of different characteristics. Thus, one could try to discover the uncertainty by the fuzzy set theory.

Among the fuzzy clustering methods, fuzzy c-means (FCM) and its variants have been developed very popular in the recent years. To compensate the shortcomings of FCM, two important aspects are usually incorporated to the original FCM, which are local spatial constraint and local gray-level relationship. Ahmed et al. [2] introduced a neighboring item into the objective function to bias the solution toward piecewise-homogeneous regions. The neighboring item is calculated by the neighboring pixels variation to its cluster center. Chen and Zhang [3] used the mean or median of neighboring pixels lying within local window to replace the pixels in the local window to simply its computation. Cai et al. [4] proposed an efficient local similarity measure by fusing local spatial and gray information for medical image segmentation. Besides, the segmenting time is only dependent on the number of the gray-levels by histogram statistics. Another two recent important literatures of FCM are Krinidas and Chatzis [5], and Gong et al. [6]. In [5,6], they present two fuzzy local neighborhood factors incorporating the spatial and gray level relationship. Additionally, the kernel distance measure is introduced into the objective function to improve its robustness to noise or outliers. Overall, considering above variants of FCM, some common features in these algorithms can be found. They use only one clustering index to guide their searching process, which may be very appropriate for particular data distribution of clusters. Hence, it is natural to consider multiple objective functions and optimize them simultaneously. Besides, the SAR images with multiplicative speckle noises are seldom involved in these algorithms.

Another important issue for SAR image segmentation lies in its vast pixels with overlapping characteristic. Taking the typical small SAR image (512 × 512) as an example, it consists of 262,144 pixels, which is great amount of samples for the traditional clustering methodologies. To this end, the watershed transformation (WT) is utilized to preliminarily partition the image into many local non-overlapping patches or super-pixels. We can see WT merges great amount of local pixels with similar intensity. Here, a gradient-based water-shed transformation (GWT) and morphological operator [7] are employed here. Currently, many similar algorithms can be used for initial segmentation or segmentation, such as mean shift based gradient vector flow [8,9], anisotropic mean shift based fuzzy c-means segmentation [10]. Here, GWT is used for merge the local similar pixels though it may create many over-segmented results. They are used for the subsequent fine segmentation framework of the paper; in other words, the shortcomings of the traditional WT can be used here.

In this paper, an efficient SAR image segmentation algorithm by using nonlocal mean filter in principal component analysis (PCA) [11] and multi-objective kernel clustering in artificial immune system [12,13] is presented. The nonlocal mean filter has been implemented in PCA by Tasdizen [11] and its performance has been tested by images corrupted with additive independent Gaussian noise with different standard deviations. Here, the performance of the technology in removing multiplicative speckle noise is also investigated. Besides, a fast strategy in selection of subspace dimensionality in PCA is given. The crucial step in SAR image segmentation is how to cluster the large amounts of highly overlapped pixels. Here, watershed transformation is utilized to over-segment the filtered image by nonlocal mean into reduced number of super-pixels or local regions. Afterwards, the principles of the vertebrate immune system’s characteristics of learning, memory and adaptability in artificial immune system (AIS) are used to construct an image segmentation methodology. This method is devised in multi-objective optimization and kernel distance measures to improve its robustness to residual noise by former filter. Moreover, the performance of proposed algorithm is also investigated by testing on two SAR images with three variants of FCM, called FGFCM [4], FLICM [5] and KWFLICM [6], and a kernel graph cut method, called KGC [14]. Besides, some involved parameters and CPU times are also presented and analyzed.

The remaining part of the study is organized as follows. Section II describes the details of the AIS-based two-objective automatic segmentation framework. The related issues in the proposed algorithm are discussed and explained. Next, the experimental evaluations of proposed algorithm are presented in Section III. Finally, we outline the conclusion of the paper.

2. Methodology of Mkcis

A very important application of SAR image segmentation is to partition an image into different types of non-overlapping land cover such that the pixels in the same type are more similar to each other than pixels in different types. Unfortunately, the speckle noises in SAR image usually bring much more uncertainty and ambiguity to the clustering process. Therefore, corresponding filtering technology is essentially required to suppress the speckle noise in SAR image. As it can be seen, the nonlocal means filter has been shown very attractive de-noising performance by exploring redundancy information of a pixel’ gray level and geometrical configuration in its local neighborhood [15–17].

Furthermore, over the past few decades there has been a growing interest in the use of biology as a source of inspiration for solving computational problems. The nature inspired algorithms have attracted lots of interests and they have become important parts of computational intelligence. Many of them have been proposed and applied to real-world problems. Among them, artificial immune system (AIS) inspired by the biological immune system is one representative of these kinds of new algorithms. It mimics the human immune system’s distinguishing characteristics of robustness, memory, highly parallelism and adaptability. AIS-based algorithms have begun to emerge as problem solvers in image classification recently and two popular used clustering indices are designed in kernel distance mapping. The algorithm can be divided into two stages, i.e., the first stage attempts to remove the speckle noise as much as possible and the second stage is used to partition the smoothed image by multi-objective kernel clustering in AIS. The basic procedure of the algorithm has shown in Figure 1 and it is called MKCIS for brief description in the subsequent paragraphs.

2.1. Nonlocal Means Filter in PCA

The redundancy information in images is adequately exploited by nonlocal means filter, which means that every small patch in an image have many similar patches in the same image. As a result, the speckle randomly distributed in the image can be suppressed by these similar local patches. If I is an SAR image with noises, x(i) is the observed image and NL(i) is the noise removed image by nonlocal means filter, the denoising operator can be defined as the weighted average of the pixels in the original image.

$$NL(i)={\displaystyle \sum {}_{j\in I}}w(i,j)x(j)$$

(1)

$$w(i,j)=\frac{1}{\yen (i)}{e}^{-\frac{{\Vert v(N{B}_i)-v(N{B}_j)\Vert }_{2,a}^2}{h^2}},\yen (i)={\displaystyle \sum {}_j}{e}^{-\frac{{\Vert v(N{B}_i)-v(N{B}_j)\Vert }_{2,a}^2}{h^2}}$$

(2)

The weights in Equation (1) measure the similarity of pixel i and their neighborhoods NB_i, v(NB_i) is the vector of neighboring pixel around i, a is the standard deviation of the Gaussian kernel, and h controls the decay of the exponential function.

Recently, the introduction of lower dimensional subspace of neighboring vectors combing with nonlocal mean (NLM) filter can save computational cost and performs more robust to noise than distances calculated in the full-dimensional space. The covariance matrix of neighboring pixel can be listed as follows.

$$C_{NB}=\frac{1}{¡}{\displaystyle \sum _{i\in ¡}\left(v(N{B}_i)-\overline{v(N{B}_i)}\right)}{\left(v(N{B}_i)-\overline{v(N{B}_i)}\right)}^T$$

(3)

¡ is the set of neighboring pixel around i. Compute the eigenvectors E = {e₁, e₂, …, e_N} and eigenvalues Φ = {λ₁, λ₂, …, λ_N} by de-compositing above matrix. If the d larger eigenvectors are selected to span d-dimensional space, the original neighboring vector in (2) can be denoted in the low dimensional space.

$$v_d(N{B}_i)={\displaystyle \sum _{i=1}^d\langle v(N{B}_i),e_i\rangle }{e}_i$$

(4)

Hence, if we use Equation (4) to replace the item v(NB_i) in Equation (2). ˂v(NB_i), e˃ denotes the inner product of the two vectors. The meaning of Equation (1) can be updated by non-local mean estimator in low-dimensional space.

How to select the dimensionality of the subspace is the essential issue of the NLM in PCA. Here, knee point detection using maximum gradient tracing to select the changing point of the curve of eigenvalues is used here [11,18]. Here, local image patches are selected and their covariance matrixes are decomposed. By choosing different number of eigenvalues to reconstruct the local patches, the differences or map of the reconstructed patches and the patches rebuilt by only one largest eigenvector are plot and maximum gradient of the knee point can be found. The main idea is shown in Figure 2.

2.2. Multi-Objective Kernel Clustering in AIS

AIS with population iteration on the level of total pixels is usually very time consuming because there are enormous number of pixels in current SAR image, even for small size of images with moderate resolution. Therefore, a preprocessing stage is usually required to partition the filtered image into local small patches or “superpixels”. To this end, the well-known gradient-based water-shed transformation (GWT) to initially partition an image into local homogeneous regions is utilized. In this study, the dilation and erosion operations in WT are both on a 3 × 3 window and about 1000 small patches are obtained in an image with 256 × 256.

AIS are such type of adaptive artificial intelligence techniques which mimics theoretical immunology and observed immune principles and models to solve complex problems. Among many immunological models, the clonal selection principal [19,20] have proved to be the sources of inspiration for many people working in the AIS-based optimization field for its simplicity and efficiency. Here, we want to present an efficient SAR image clustering algorithm by using the features of clonal proliferation, affinity maturation, and clonal selection of human immune system. Besides, two simply clustering indices are designed in kernel distance mappings to lead the searching process. The definition of the two indices can be described in the following equations.

$$X{B}^{kernel}=\frac{{\displaystyle {\sum }_{k=1}^K{\displaystyle {\sum }_{i,j=1}^n{u}_{kj}^2\left(1-\wp (x_j,z_k)\right)}}}{n{\min }_{i,j}\left(1-\wp (z_i,z_j)\right)},{u_{kj}=1/{\displaystyle {\sum }_{i=1}^K\left(\frac{\Vert z_k-x_j\Vert }{\Vert z_k-x_i\Vert }\right)}}^2$$

(5)

$$Con{n}^{kernel}={\displaystyle {\sum }_{i=1}^n{\displaystyle {\sum }_{j=1}^{L}v(x_{i,j}),}\text{where}v(x_{i,j})=\{\begin{array}{l}\raisebox{1ex}{$\left(1-\wp (x_i,z_j)\right)$}\!\left/ \!\raisebox{-1ex}{$j$}\right.,:\exists C_m:x_i\in C_m\Lambda x_j\in C_m\\ 0,\text{otherwise}\end{array}}$$

(6)

Note that x_i, i = 1, 2, …, N is a sample in a clustering problem and z_p, p = 1, 2, …, K is the cluster center, n is the total number of samples, K is the number of categories, U_{K × n} is the fuzzy membership matrix and L is the parameter to determine the number of neighbors that contribute to the connectivity measure. It is clear that XB^kernel is formulated by calculating the ratio of the summation of variation to the minimum separation, and the lower values of the index can provide better partition of image. Conn^kernel is defined by dividing Euclidian distance of datum i to its j-th nearest neighbor by gradually decreasing penalty factor. Note that K is the number of clusters and n is the total pixels.

As we know, a non-linear function in kernel method can be used to map the samples from the input space to a high dimensional feature space. Accordingly, the complicated mixed distribution of pixels in SAR images can be further exploited. In kernel distance measure between the pixel x_i and x_j can be calculated through the kernel function $\wp (x_i,x_j)=\exp \left(-{\Vert (x_i)-(x_j)\Vert }^2/(2{\sigma }^2)\right)$ in the input space, where σ > 0, and it can be represented in the following equation, φ(x_i) is the value of the features of the pixel x_i.

$$\begin{array}{l}{\Vert \phi (x_i)-\phi (x_j)\Vert }^2={\left(\phi (x_i)-\phi (x_j)\right)}^T\left(\phi (x_i)-\phi (x_j)\right)\\ =\phi {(x_i)}^T\phi (x_i)-2\phi {(x_i)}^T\phi (x_j)+\phi {(x_j)}^T\phi (x_j)\\ =\wp (x_i,x_i)-2\wp (x_i,x_j)+\wp (x_j,x_j)\end{array}$$

(7)

Here, the widely used Gaussian RBF kernel to substitute the item $\mathrm{p}(\mathrm{L},\mathrm{L})$ in above equation is employed, which can be simplified to the following equations.

$$\begin{array}{l}{\Vert \phi (x_i)-\phi (x_j)\Vert }^2=\wp (x_i,x_i)-2\wp (x_i,x_j)+\wp (x_j,x_j)\\ =2\left(1-\wp (x_i,x_j)\right)\\ \text{where},\wp (x_i,x_j)=\exp \left(-{\Vert (x_i)-(x_j)\Vert }^2/(2{\sigma }^2)\right)\end{array}$$

(8)

In this way, the pixels in original space can be transformed into non-Euclidean distance measures. Thus, the nonlinear structure of pixel distribution in the low-dimensional space can be separated in the high-dimensional space by kernel trick.

It has been discussed in the literatures [13,21] that multi-objective clustering (MOC) can partition more types of data sets with complicated and diverse characteristics than single-objective clustering (SOC). Here, the pseudo-code of the proposed algorithm MKCIS is shown in

Table 1. Pseudo-Code of mkcis.

Input: SAR image X = {x1, x2, …, xn}, number of clusters K, size of searching window in NLM w, number of population N; size of clonal pool C;
Step1. Perform NLM-PCA(w) filter on the input SAR image X.
Step2. Implement the watershed transformation on the filted image and the raw segmentation results of can be denoted by X1.
Step3. i=0, X1 is the input local image patches, initialize population P(i) randomly and compute its two objective functions F(i) in kernel method defined in equation. (5)–(6).
[Minmal(i)] = FindBestSolution(F(i), P(i));
Label = 0;
Step4. while Label < M
[A(i), AF(i)] = FindSmallBestIndividuals(P(i), F(i), C); %find C the better solutions
[NP(i)] = ProportionalClone(A(i), AF(i), N); %perform the clone on above selected solutions
[NP(i)] = Crossover(NP(i), A(i)); %perform gene crossover on the above new solutions
[NP(i)] = Mutation([NP(i)]); %perform gene perturbation (local search) on the above new solutions
[NF(i)] = FitnessCalculation(NP(i)); %judge the above new created solutions
TotalP = NP(i) ∪ P(i); %merge the two-consecutive-steps solution sets at variable space
TotalF = NF(i) ∪ F(i); % merge the two-consecutive-steps solution sets at objective space
[P(i), F(i)] = UpdatePopulation(TotalP, TotalF); % update the current solutions
I = i+1;
[Minmal(i)] = FindBestSolution(F(i), P(i)); % find the current best solution
if |Minmal(i) − Minmal(I + 1)| < α
Label = Label + 1;
end-if
end-while.
Step5. find the best soltion among the final non-dominated front. Output k clusters of X.

. It can be seen that Step 1 and Step 2 are the nonlocal means filters in low-dimensional space and watershed transformation respectively. Step 3 and Step 4 are the procedure of multi-objective clustering using kernel distance mapping and clonal selection algorithm. The proportional clone is utilized in the method to calculate the antibodies’ computer resource to implement clonal proliferation in AIS. Besides, crowing-distance [22] is employed to measure the sparsity at local solution space. The antibody with larger value of crowing-distance is given more chances to search in less-crowded regions of the trade-off front. Additionally, since simulated binary crossover and polynomial mutation were adopted many times in current evolutionary multi-objective optimization algorithm (EMOA) literatures [22,23]. Here, we continue to use them for affinity maturity operation. Finally, the population updating strategy or the clonal selection is designed by the above crowing-distance to select the antibodies locating at the less-crowded regions of the trade-off front. Thus, the current local un-researching areas in solution space can be fully exploited by the strategy.

3. Experimental Study

In this section, the experimental results of the proposed algorithm on two complicated images with multiplicative speckle noise are demonstrated. Additionally, three recently proposed variants of FCM: KWFLICM, FLICM, FGFCM, and a kernel cut graph, called KGC, are utilized to compare its efficiency and robustness. The specific parameters settings of MKCIS are presented here. For, gradient-based watershed transformation is employed here, the structure elements matrix is a 3 × 3 window and about 1000 small patches are obtained in the two image. For NLM-PCA, the size of local similarity window in NLM is 3 × 3 and that of searching window is 7 × 7, and the parameter of the bandwidth uses a fast bandwidth selection by standard-variance of each current cluster. For MOC, the number of population is 40, the size of clone pool is 20, the crossover probability is 0.9 and mutation probability is 0.2, which are the recommended values by the published literatures [22,23]. Furthermore, the influence of the number of eigenvectors used in PCA and the different number of looks in contaminating the synthetic SAR images are investigated in the subsequent experimental study.

3.1. Experimental Results Presentation and Discussion

The first experiment is tested on a synthetic SAR image with eight classes and 260 × 260 pixels, as shown in Figure 3a. The artificial SAR image with three-looks follows the Goodman’s multiplicative speckle noise model [24]. It is clear that the three variants of FCM cannot achieve good segmentation results of the image. There are lots of misclassified spots at different patches in the image. The reason may be that the similarity measures designed by incorporating local spatial and gray information cannot produce enough suppressing results of the speckle noise. It seems that KGC in Figure 3e can recognize most classes of the synthetic image, however, two white local regions at the right down of the image are not well separated, i.e., they are seriously mixed together. Besides, the positive factor α to adjust the weight of the prior information is hard to set to an appropriator value for different tested problems. In fact, the proposed algorithm seems to have presented relatively better segmentation result not only in region consistence but also in the boundary localization.

In addition, the segmentation results of the five algorithms over repeated independent runs are also tested here.

Table 2. Statistical segmentation results of accurate rate (AR) and entropy (E) of the synthesized SAR image over 30 independent runs.

T	KWFLICM	FLICM	FGFCM	KGC	MKCIS
AR	85.12 (0)	65.28 (0.52)	78.15(0)	78.95(24.5)	98.14(0.01)
E	2.58(0.01)	2.59 (0.01)	2.59(0)	2.59(0.15)	2.55(0.2)

shows the statistical mean and standard deviation of segmentation results over 30 independent runs. Besides, the index: accurate rate (AR) and an information theory induced index, called entropy (E) are used to calculate the clustering performance for the algorithms. The statistical results in Table 2 also agree with the visually inspection in Figure 3.

Another experiment is carried out by using a real SAR image from the website of German Aerospace Center. It has 1-m spatial resolution at X-band of Swabian Jura, Germany by Terra-SAR. The image with 280 × 280 pixels consists of four distinct typical land covers: urban villages and four types of crops.

The segmentation results of the five algorithms in segmenting the real SAR image are shown Figure 4 and the statistical segmentation results over many repeated independent runs are listed in

Table 3. Statistical segmentation results of entropy (E) of the real SAR image over 30 independent runs.

Index	KWFLICM	FLICM	FGFCM	KGC	MKCIS
E	2.57(0.01)	2.78 (0.01)	2.57(0)	2.55(0.03)	2.55(0.01)

. Note that the ground truth of SAR images are usually absent or difficult to obtain, therefore, the statistical results of accuracy rate are not listed in Table 3. It is clear that KGC and MKCIS have achieved the better segmentation results that other three algorithms. And the similar phenomenon is also shown in statistical results in Table 3.

3.2. Sensitivity to Different Looks of Speckle Noise

The major difficulty of SAR image segmentation is the large amounts of unpredictable and uncertain speckle noise in this kind of image. To evaluate the robustness of the proposed algorithm under the effect of noise, the synthetic SAR image is contaminated by different number of looks of simulated speckle noise. In general, the larger number of looks is and the less amount of speckle noise is. The segmenting results of the synthetic SAR image with 1-look to 10-look results by five algorithms are shown in Figure 5. The curves show that the mean segmentation accuracies of five algorithms ascend in a steady style with different number of looks. Besides, the performances of KGC and MKCIS are nearly the same when the number of look is more than eight. Besides, it is obvious that all the mean values of partitioning accuracy of MKCIS are greater than 90%.

3.3. Influence of the Number of Eigenvectors in PCA

Since NLM in PCA is used, it is intuitive to study the behavior of NLM with respect to different number of subspace dimensionality, which is shown in Figure 6. Note that if the number of dimensionality is equal to the number of pixels in local similarity window, NLM + PCA will degenerate to NLM. Figure 6 tells us that if the dimensionality is less adopted, the segmentation accuracy of MKCIS will decline slowly. The reason may be that the increased accuracy at lower values is that kernel distances calculated in the lower dimensional space are likely to be more accurate and robust than distance computed from the full-dimensional space because PCA discards the most irrelevant dimensions. In our study, about two or three eigenvectors are selected to implement the subspace projections. Clearly, about 98% of segmentation accuracy is obtained.

3.4. Running Times Comparison and Analysis

The running times of the five times are also studied here. Note that FLICM and KGC are programmed in C and the other three are in MATLAB 7.9. And the experimentations are implemented at Lenovo workstation Think-Centre M8400t (3.40 GHZ, 4 GB RAM; Intel (R) Core(TM) i7-3770CPU). In Figure 7, it can be seen that KWFLICM needs more CPU running times (about 500 seconds), which is same to its original literature. FGFCM takes the least computational times (about 10 seconds). The times of MKCIS are about 50 s (15 s for the Step 1 and Step 2, 35 s for the Step 3 and Step 4 of MKCIS in Table 1). Thought, MKCIS is not the fastest algorithms among the five ones, it can supply the best segmentation performance among them. Thus we expect that it can provide an effective option for SAR imagery segmentation.

Finally, Figure 8 illustrates the computational cost on the images with different size for MKCIS. The proposed algorithm is computational consuming when the number of image pixels increases, since the covariance matrixs are required to decompose in NLM in each iteration step. The computational burdens of Step 3 and Step 4 of MKCIS lies in the merged local images patches, not the original number of image pixels; therefore, though the size of synthetic SAR images increase, the computer cost of Step 3 and Step 4 of MKCIS grow linearly with increase of image size.

4. Conclusions

An efficient unsupervised multi-objective kernel classifier using non-local means filters in PCA and clonal selection theories in AIS for SAR image segmentation is developed. The experimental study is designed by using two SAR image and comparing with four the-state-of-art image segmentation algorithms, and it shows that the proposed algorithm has obtained promising segmentation result. Besides, some of involved critical parameters are investigated also. As we know, a desirable search strategy should lead the population from multiple directions or paths; since the prior knowledge about the location of the global optimal solution are absent. The multi-objective kernel clustering proposed in this study is just a simple way to use multi-objective optimization. We believe that there should be more suitable multi-objective framework for SAR image segmentation.

As far as we know, the searching process of segmentation algorithms, no matter in single-objective and multi-objective, are essentially guided by the objective-functions. Thus, the clustering validity indices are the key technique in image segmentation, which is worthy of further study. Usually, there are several items in a clustering validity index; do the items in the index have equal importance? Is it reasonable to normalize each item before calculating the whole index? Furthermore, the investigation of the combinatorial properties of the current popular clustering validity indices is required to further study. It is possible that, although some individual indices do not yield the best segmentation accuracy, their combinations in pairs or groups perhaps provide more suitable partitions. The authors are currently working in this direction.