Affinity Propagation Based on Structural Similarity Index and Local Outlier Factor for Hyperspectral Image Clustering

Abstract

In hyperspectral remote sensing, the clustering technique is an important issue of concern. Affinity propagation is a widely used clustering algorithm. However, the complex structure of the hyperspectral image (HSI) dataset presents challenge for the application of affinity propagation. In this paper, an improved version of affinity propagation based on complex wavelet structural similarity index and local outlier factor is proposed specifically for the HSI dataset. In the proposed algorithm, the complex wavelet structural similarity index is used to calculate the spatial similarity of HSI pixels. Meanwhile, the calculation strategy of the spatial similarity is simplified to reduce the computational complexity. The spatial similarity and the traditional spectral similarity of the HSI pixels jointly constitute the similarity matrix of affinity propagation. Furthermore, the local outlier factors are applied as weights to revise the original exemplar preferences of the affinity propagation. Finally, the modified similarity matrix and exemplar preferences are applied, and the clustering index is obtained by the traditional affinity propagation. Extensive experiments were conducted on three HSI datasets, and the results demonstrate that the proposed method can improve the performance of the traditional affinity propagation and provide competitive clustering results among the competitors.

1. Introduction

Hyperspectral image (HSI) has gradually become a powerful tool with its rich spectral and spatial information, which is widely used in environmental monitoring, fine agriculture, mineral exploration, military targets, and many other fields [1,2,3]. Though the potentialities of hyperspectral technology appear to be relatively wide, the analysis and treatment of these data remain insufficient [4]. Classification is an important manner in which to exploit HSI, which can be divided into supervised classification and unsupervised classification. Compared with the supervised classification, the unsupervised classification, also known as clustering, can automatically detect the distinct classes in an objective way without training samples. In fact, training samples are very difficult to access for some applications. As a result, it is meaningful to study the clustering technology.

Thus, in this paper, we mainly focused on the clustering approach for HSI partitioning. Generally speaking, clustering techniques can be mainly categorized into nine main types [5]: centroid-based clustering [6,7,8], density-based clustering [9,10,11], probability-based clustering [12,13,14,15], bionics-based clustering [16,17], intelligent computing-based clustering [18,19], graph-based clustering [20,21], subspace clustering [22,23,24,25], deep learning-based clustering [26,27,28], and hybrid mechanism-based clustering [29,30]. Affinity propagation (AP) [31] is a centroid-based clustering method that identifies a set of data points that best represent the dataset and assigns each data point to a single exemplar. Compared with classical centroid-based clustering, AP is insensitive to the initial centers and outliers. Therefore, it is widely used in face recognition [32], HSI classification [33], fault detection [34], and many other fields [35,36,37]. At the same time, many scholars have carried out in-depth studies of AP and put forward improved versions. The measuring of the similarity is a topic that has received a lot of attention. Wan, X. J. et al. [38] used dynamic time warping to measure the similarity between the original time series data and obtained the similarity between the corresponding components, which was applied to cluster the multivariate time series data with AP. Wang, L. M. et al. [39] proposed a novel structural similarity to solve the unsatisfactory clustering impact of AP when dealing with complex structural datasets. Zhang, W. et al. [40] applied soft scale invariant feature transform to adopt the similarity between any pair of images to clustering the images by AP. Qin, Y. et al. [41] integrated the spatial-spectral information of HSI samples into non-negative matrix factorization for affinity matrix learning of HSI clustering. Fan, L. et al. [42] proposed a local density adaptive affinity matrix, which embeds both spectral and spatial information and uses it for HSI clustering. We can see that these algorithms use tailored similarity for datasets to increase the performance of the clustering methods. Meanwhile, many studies have aimed at optimizing the exemplar preference of AP. Chen, D. W. et al. [43] defined a novel stability measure for AP to automatically select the appropriate exemplar preferences. Gan, G. J. and M. K. P. Ng [44] proposed a subspace clustering algorithm by introducing attribute weights in the AP. The new step could iteratively update the exemplar preferences to identify the subspaces in which clusters are embedded. Li, P. et al. [45] proposed a modified AP named as adjustable preference affinity propagation, which initials the value of preferences according to the data distribution. In addition, the convergence speed [46] and the calculation scale [4] are also mentioned in the literature.

However, the application of AP for HSI analysis is still insufficient. The reasons are mainly twofold: (1) the complex spectral structure of the HSI dataset; and (2) the usage of the spatial information of the HSI dataset. As a result, it is meaningful to apply a spatial-spectral strategy to extract information from the HSI dataset, which can better express the similarity between samples. Moreover, it is also an interesting question to modify the value of the exemplar preference based on the characteristics of the HSI dataset.

The structural similarity (SSIM) index [47] was proposed as a promising metric of image, which accounts for spatial correlations. Aside from the mean intensity and contrast, the structural information of an image is described as those attributes that represent the structures of the objects in the visual scene. The complex wavelet SSIM index [48], also called CW-SSIM, is a type of SSIM in the spatial and complex wavelet domains. The structure information is represented by the spatial distribution of grayscale values as well as the magnitude and phase responses of the multidirectional Gabor filters. The CW-SSIM has been shown to be a useful measure in image quality assessment [49,50], feature extraction [51], and anomaly detection [52]. On the other hand, the local outlier factor (LOF) [53] was first used in the noisy detection in HSI analysis [54]. The calculation of LOF is related to a restricted local region around an object [55]. Yu, S. Q. et al. [56] proposed a low-rank representation in the field of hyperspectral anomaly detection, which facilitates the discrimination between the anomalous targets and background by utilizing a novel dictionary and an adaptive filter based on the local outlier factor. Few studies can be found that have applied CW-SSIM and LOF to express the similarity of samples in HSI datasets and generate the similarity matrix of AP. We applied the strategy of the CW-SSIM as well as the LOF to act on the traditional pixel-based similarity metric and the exemplar preference, respectively, and used them in AP for HSI clustering.

In this paper, an improved AP with CW-SSIM and LOF (CLAP) is presented based on the properties of hyperspectral data. The metrics of the similarity and the exemplar preference are both of concern in the proposed CLAP. Unlike the traditional approach that only calculates the pixel-based spectral similarity [57] to generate the similarity matrix of AP, the CLAP claims to combine the structure-based spatial similarity with pixel-based spectral similarity. In the proposed algorithm, the CW-SSIM is used to extract the structure-based spatial similarity of the HSI samples. To reduce the computational effort, we used principal component analysis (PCA) to reduce the sample size, and defined a novel strategy to reduce the computational complexity. Specifically, PCA was used to pre-process the hyperspectral data, and the $n$ dimensions of the highest explained principal components were reserved to extract the spatial information. We extracted the spatial neighborhood blocks of the samples on each principal component (PC) and recomposed new sample sets separately. After that, to reduce the computational complexity, the average of the samples for each principal component sample set was calculated, which was named Average, and the CW-SSIM was applied to calculate the similarity between samples and the corresponding Average. Then, the results were used to generate the spatial similarity matrix ($S_{\mathrm{cw}}$). Finally, the final similarity matrix ($S_{\mathrm{f}\left(i\ne k\right)}$) was obtained by the pixel-based spectral similarity matrix ($S_{\mathrm{pix}\left(i\ne k\right)}$) and the structure-based spatial similarity matrix ($S_{\mathrm{cw}}$). Meanwhile, unlike the traditional definition of the consistent exemplar preferences that are directly based on the minimum (mean) value of the similarity, the CLAP uses the LOF to generate the weights to revise these consistent exemplar preferences. The key idea behind this is that the local neighborhood density of the cluster center should be uniform and smooth, according to the manifold assumption. Specifically, we first calculated the LOFs for all samples. Then, these LOFs were used to obtain the smoothness coefficients ($L_{\mathrm{sm}}$) by a self-defined formula, which represent the degree of the uniformity and smoothness of the local neighbourhood density of the samples on the spectral space. The LOF coefficients ($L$) were obtained using the smooth coefficients ($L_{\mathrm{sm}}$) and applied as weights to weight the original consistent preferences ($S_{\mathrm{pix}\left(i=k\right)}$) to obtain the final exemplar preferences ($S_{\mathrm{f}\left(i=k\right)}$). Finally, $-S_{\mathrm{f}}$ was used as the similarity matrix of AP and the clustering index of the samples were obtained by AP clustering. The flow chart of the proposed CLAP is shown in Figure 1. The novel contributions in the proposed method are as follows:

1.

New spatial-spectral similarity metrics for the hyperspectral dataset were defined and applied to AP clustering.

2.

The CW-SSIM was used to measure the similarity of the HSI samples and a new computational strategy was defined to reduce the computational effort.

3.

The LOF was used to define the degree of the uniformity and smoothness of the local neighborhood density of a sample and applied to revise the exemplar preference of AP.

In this study, we used three hyperspectral datasets as benchmarks to compare the proposed method to traditional clustering methods. Experiments showed that the proposed method outperformed the competition.

The rest of this paper is organized as follows. Section 2 presents the related work and the proposed method. The experimental results and the discussions are presented in Section 3 and Section 4. Finally, Section 5 presents some conclusions and our future work.

2. Method

2.1. Affinity Propagation

The AP is a clustering algorithm based on the exemplar method. It finds a set of data points to exemplify the data, and associates each data point with one exemplar. Specifically, given the samples ${\mathit{x}}_i,{\mathit{x}}_k\in R^d,i,k=1,2,\cdots ,N$, $d$ is the dimension, $N$ is the number of data points, the AP first computes a similarity matrix $\mathit{s}$ of all samples, which is defined as:

$$s\left({\mathit{x}}_i,{\mathit{x}}_k\right)=-\left|\left|{\mathit{x}}_i,-,{\mathit{x}}_k\right|\right|{}^2 i\ne k$$

(1)

$$s\left({\mathit{x}}_i,{\mathit{x}}_i\right)=\mathit{min}\left(s\right) i\in \left(1,N\right)$$

(2)

where $s\left({\mathit{x}}_i,{\mathit{x}}_k\right)$ is the element of the similarity matrix and is defined by the opposite of the squared Euclidean distance. $s\left({\mathit{x}}_i,{\mathit{x}}_i\right)$ is the diagonal element of the similarity matrix and is called the exemplar preference, which is set to the minimum value of the similarity matrix. It represents the prior suitability of a data point to be the exemplar, and controls the number of the clusters of AP. Then, the AP exchanges messages between data points, which are named responsibility $\mathit{r}$ and availability $\mathit{a}$.

$$r\left(i,k\right)=s\left({\mathit{x}}_i,{\mathit{x}}_k\right)-ma{x}_{j,j\ne k}\left\{a\left(i,j\right)+s\left({\mathit{x}}_i,{\mathit{x}}_j\right)\right\}$$

(3)

$$a\left(i,k\right)=min\left\{0,r\left(k,k\right)+{\displaystyle \sum }_{j,j\ne \left\{i,k\right\}}max\left\{0,r\left(j,k\right)\right\}\right\}$$

(4)

$$a\left(k,k\right)={\displaystyle \sum }_{j,j\ne k}max\left\{0,r\left(j,k\right)\right\}$$

(5)

where $r\left(i,k\right)$ and $a\left(i,k\right)$ are the elements of $\mathit{r}$ and $\mathit{a}$, and are initialized to 0. To avoid the oscillations, $\mathit{r}$ and $\mathit{a}$ are damped as:

$${\mathit{r}}_{t+1}=\lambda {\mathit{r}}_{t-1}+\left(1-\lambda \right)$$

(6)

$${\mathit{a}}_{t+1}=\lambda {\mathit{a}}_{t-1}+\left(1-\lambda \right){\mathit{a}}_t$$

(7)

where $\lambda$ is the factor of damping, which satisfies $0.5\le \lambda \le 1$, and $t$ is the number of the iteration. Finally, the exemplar vector $\left(c\right)$ can be obtained as:

$$c\left(i\right)=ma{x}_k\left(a\left(i,k\right)+r\left(i,k\right)\right)$$

(8)

The AP is converged if the iterative number exceeds the predetermined value or the exemplar vector remains unchanged for some constant iterations.

2.2. Complex Wavelet Structural Similarity

The CW-SSIM is a type of structural similarity index based on local phase measurements in the spatial and complex wavelet domain, which is designed to coincide with the human perceptual system and could provide a good approximation of perceptual image quality [48,51,58]. The CW-SSIM index is designed to separate the phase from luminance distortion measurements and simultaneously insensitive to luminance and contrast changes. Specifically, a complex version [59] of the steerable pyramid transform is first applied to decompose the compared two images [60]. The complex wavelet coefficients are expressed as ${\mathit{c}}_x=\left\{c_{x,i}|i=1,\cdots ,I\right\}$ and ${\mathit{c}}_y=\left\{c_{y,i}|i=1,\cdots ,I\right\}$. The CW-SSIM index is defined as:

$$D_{\mathrm{CW}}\left(c_{x,i},c_{y,i}\right)=\frac{2\left|{{\displaystyle \sum }}_{i=1}^I{c}_{x,i}{c}_{y,i}^{*}\right|+K}{{{\displaystyle \sum }}_{i=1}^I{\left|c_{x,i}\right|}^2+{{\displaystyle \sum }}_{i=1}^I{\left|c_{y,i}\right|}^2+K}$$

(9)

where $c^{*}$ is the complex conjugate of $c$, and $K$ is a small positive stabilizing constant. The value of the index ranges from 0 to 1, where 1 implies no structural distortion.

2.3. Local Outlier Factor

The LOF is an outlier index that indicates the degree of the outlier-ness of a sample [53,55]. The LOF exploits the density information from the local neighborhood of each sample in the feature space. Specifically, given a dataset $D$, for any positive integer $k_{\mathrm{lof}}$, the k-distance of sample $\mathit{x}$, denoted as $di{s}_{\mathrm{k}}{}_{\mathrm{lof}}\left(\mathit{x}\right)$, is defined as the distance $d\left(\mathit{x},\mathit{o}\right)$ between $\mathit{x}$ and $\mathit{o}\in D$ so that:

1.

For at least $k$ samples ${\mathit{o}}^{\prime }\in D\backslash \left\{\mathit{x}\right\}$, it holds that $d\left(\mathit{x},{\mathit{o}}^{\prime }\right)\le d\left(\mathit{x},\mathit{o}\right)$, and

2.

For at most $k-1$ objects ${\mathit{o}}^{\prime }\in D\backslash \left\{\mathit{x}\right\}$, it holds that $d\left(\mathit{x},{\mathit{o}}^{\prime }\right)<d\left(\mathit{x},\mathit{o}\right)$.

Next, given the k-distance of $\mathit{x}$, the k-distance neighborhood of $\mathit{x}$ contains every object whose distance from $\mathit{x}$ is not greater than the k-distance, in other words

$$N_{{\mathrm{k}}_{\mathrm{lof}}}\left(\mathit{x}\right)=\left\{\mathit{q}\in D\backslash \left\{\mathit{x}\right\} | d\left(\mathit{x},\mathit{q}\right)\le dis\_k_{\mathrm{lof}}\left(\mathit{x}\right)\right\}$$

(10)

where the sample $\mathit{q}$ is called the k-nearest neighbor of $\mathit{x}$ and is designated as $N_{{\mathrm{k}}_{\mathrm{lof}}}\left(\mathit{x}\right)$. Next, the reachability distance of sample $\mathit{x}$ with respect to sample $\mathit{o}$ is defined as:

$$reach\_dis{k}_{\mathrm{lof}}\left(\mathit{x},\mathit{o}\right)=\max \left\{dis\_k_{\mathrm{lof}}\left(\mathit{o}\right),d\left(\mathit{x},\mathit{o}\right)\right\}$$

(11)

Then, the local reachability density of $\mathit{x}$ is defined as:

$$lr{d}_{{\mathrm{k}}_{\mathrm{lof}}}\left(\mathit{x}\right)=\frac{1}{\left(\frac{{{\displaystyle \sum }}_{o\in N_{{\mathrm{k}}_{\mathrm{lof}}}\left(\mathit{x}\right)}reach\_dis{k}_{\mathrm{lof}}\left(\mathit{x},\mathit{o}\right)}{\left|N_{{\mathrm{k}}_{\mathrm{lof}}}\left(\mathit{x}\right)\right|}\right)}$$

(12)

where $\left|N_{{\mathrm{k}}_{\mathrm{lof}}}\left(\mathit{x}\right)\right|$ is the number of the k-nearest neighbors of $\mathit{x}$. Finally, the local outlier factor of $\mathit{x}$ is defined as:

$$LO{F}_{{\mathrm{k}}_{\mathrm{lof}}}\left(\mathit{x}\right)=\frac{{{\displaystyle \sum }}_{o\in N_{{\mathrm{k}}_{\mathrm{lof}}}\left(\mathit{x}\right)}\frac{lr{d}_{{\mathrm{k}}_{\mathrm{lof}}}\left(\mathit{o}\right)}{lr{d}_{{\mathrm{k}}_{\mathrm{lof}}}\left(\mathit{x}\right)}}{\left|N_{{\mathrm{k}}_{\mathrm{lof}}}\left(\mathit{x}\right)\right|}$$

(13)

The value of the LOF index ranges from $1/\left(1+\epsilon \right)$ to $\left(1+\epsilon \right)$, where $\epsilon$ is a positive real number. For most $\mathit{x}$ that are deeply inside the cluster, the LOF of $\mathit{x}$ is approximately equal to 1.

2.4. Affinity Propagation Based on Structural Similarity Index and Local Outlier Factor

From the description of the AP, we can see that the similarity matrix $s$ has a great influence on the algorithm. The element $s\left({\mathit{x}}_i,{\mathit{x}}_k\right), i\ne k$ demonstrates the similarity of a pair of samples, and directly participates in the message being exchanged between samples. Both experience and experiments show that it can effectively affect the performance of AP by changing the measurement of the similarity matrix. The $s\left({\mathit{x}}_i,{\mathit{x}}_k\right),i=k$ is referred to as “preference”, which can influence the number of identified exemplars. Additionally, the sample with a larger value of preference is more likely to be chosen as the exemplar. In fact, the number of identified exemplars is not only influenced by the values of the input preferences, but also emerges from the message-passing procedure.

Based on the above analysis, we proposed a modified AP with CW-SSIM and LOF. Unlike the classical AP, the proposed CLAP introduces CW-SSIM to help generate the similarity matrix, and applies the LOF to revise the original preferences. In fact, the CW-SSIM index is used to extract the spatial similarity, and the LOF is preformed to calculate the possibility of a sample to be the exemplar. To be specific, we first discuss the usage of the CW-SSIM index in the proposed algorithm. Given the samples $x_i,{\mathit{x}}_k\in R^d,i,k=1,2,\cdots ,N$, where $d$ is the total number of the spectral dimensions and $N$ is the number of data points, we obtain the spatial neighbourhood blocks of the samples in principal component space by PCA, which are named $I_i^m$ and $I_k^m$, $m=1,2,\cdots ,n$, where $m$ is the current spectral dimension and $n$ is the total number of spectral dimensions after PCA. The $I_i^m$ and $I_k^m$ are $k_w\times k_w$ pixel images centered by ${\mathit{x}}_i,{\mathit{x}}_k$, respectively, and are extracted in the $n$ dimensions of the highest explained principal components, and $k_w$ is the side length of the window. The CW-SSIM index of the $I_i^m$ and $I_k^m$ is calculate by Equation (9), and is expressed as $S{\prime }_{\mathrm{cw}}^m\left(I_i^m,I_k^m\right)$. Assuming $p_m$ is the explained percentage of the $m$ dimension, $S_{\mathrm{pix}}\left({\mathit{x}}_i,{\mathit{x}}_k\right), i\ne k$ is the pixel distance of the samples, and the fusion distance can be expressed as:

$$S_{\prime }\mathrm{f}\left({\mathit{x}}_i,{\mathit{x}}_k\right)=S_{\mathrm{pix}}\left({\mathit{x}}_i,{\mathit{x}}_k\right)+\alpha {\displaystyle \sum }_{m=1}^n{p}_m\times \left(1-S_{\mathrm{cw}}^{\prime m}\left(I_i^m,I_k^m\right)\right)$$

(14)

where $\alpha$ is the weight coefficient to control the influence of the total CW-SSIM index, which is in the range $\left[0,1\right]$; and $p_m$ indicates that the higher explained percentage will give a greater proportion among the CW-SSIM indices.

Suppose that the computational complexity of the CW-SSIM index is expressed as $O\left(S_{\mathrm{cw}}\right)$, the computational complexity of a $N$ scale date set can be expressed as $O\left(N^2\times n\times O\left(S_{\mathrm{cw}}\right)\right)$, which is an enormous amount of computation. To alleviate the computational complexity, we propose a novel scheme to solve this problem. To be specific, the average of the neighborhood blocks of the samples can be calculated as:

$$I^m=\frac{1}{N}{\displaystyle \sum }_{i=1}^N{I}_i^m$$

(15)

We defined the modified CW-SSIM index of ${\mathit{x}}_i,{\mathit{x}}_k$ as:

$$S_{\mathrm{cw}}^m\left(I_i^m,I_k^m\right)=\left|S_{\mathrm{cw}}^m\left(I_i^m,I^m\right)-S_{\mathrm{cw}}^m\left(I_k^m,I^m\right)\right|$$

(16)

From Equation (16), we can see that the value of the $S_{\mathrm{cw}}^m\left(I_i^m,I_k^m\right)$ tends to be 0 when $I_i^m$ and $I_k^m$ have the same spatial structures. The $I^m$ acts as a constant term in the formula, and is obtained adaptively. The modified fusion distance can be expressed as:

$$S_{\mathrm{f}}\left({\mathit{x}}_i,{\mathit{x}}_k\right)=S_{\mathrm{pix}}\left({\mathit{x}}_i,{\mathit{x}}_k\right)+\alpha {\displaystyle \sum }_{m=1}^n{p}_m\times S_{\mathrm{cw}}^m\left(I_i^m,I_k^m\right)$$

(17)

where the term $\left(1-S_{\mathrm{cw}}^{\prime m}\left(I_i^m,I_k^m\right)\right)$ in Equation (14) is replaced by $S_{\mathrm{cw}}^m\left(I_i^m,I_k^m\right)$. In practice, the $S_{\mathrm{pix}}\left({\mathit{x}}_i,{\mathit{x}}_k\right)$ and the $S_{\mathrm{cw}}^m\left(I_i^m,I_k^m\right)$ are both normalized to [0,1] to avoid the dimension problem. Suppose the computational complexity of the subtraction is $O\left(1\right)$, the computational complexity of the modified CW-SSIM index in an $N$ scale dataset can be estimated as $O\left(N\times n\times O\left(S_{\mathrm{cw}}\right)+N^2\times n\times O\left(1\right)\right)$. We can see that the computational complexity is effectively reduced in the modified CW-SSIM index.

The usage of the LOF is discussed as follows. Given the sample ${\mathit{x}}_i\in R^d, i=1,2,\cdots ,N$, $d$ is the dimension, $N$ is the number of data points, and the LOF is obtained by the Equation (13), which is expressed as $LOF\left({\mathit{x}}_i\right)$. It can express the smoothness of the local density of the sample in the spectral space. The local density of the sample ${\mathit{x}}_i$ is smooth when the $LOF\left({\mathit{x}}_i\right)$ is close to 1. According to the manifold principle, the exemplars are deeply inside the clusters and their local densities are smooth. We can define the smoothness coefficient as:

$$L_{\mathrm{sm}}\left({\mathit{x}}_i\right)=e^{\left|LOF\left({\mathit{x}}_i\right)-1\right|}$$

(18)

where the $L_{\mathrm{sm}}\left({\mathit{x}}_i\right)$ is in the range of 1 to positive infinity, which denotes the smoothness of the sample ${\mathit{x}}_i$. If $L_{\mathrm{sm}}\left({\mathit{x}}_i\right)=1$, the local density of ${\mathit{x}}_i$ is completely smooth. Because the similarity matrix of the AP takes negative values of the similarity, a smaller smoothing coefficient here indicates a higher degree of smoothness. We can define the LOF coefficient of ${\mathit{x}}_i$ as:

$$L\left({\mathit{x}}_i\right)=1+\beta L_{\mathrm{sm}}\left({\mathit{x}}_i\right)$$

(19)

where $\beta$ is in the range $\left[0,1\right]$ and is the weight coefficient to control the influence of $L_{\mathrm{sm}}\left({\mathit{x}}_i\right)$. Suppose the exemplar preference of ${\mathit{x}}_i$ is expressed as $S_{\mathrm{pix}}\left({\mathit{x}}_i,{\mathit{x}}_i\right)$. The modified preference of ${\mathit{x}}_i$ can be defined as:

$$S_{\mathrm{f}}\left({\mathit{x}}_i,{\mathit{x}}_i\right)=L\left({\mathit{x}}_i\right)\times S_{\mathrm{pix}}\left({\mathit{x}}_i,{\mathit{x}}_i\right)$$

(20)

From Equation (19), we can see that if the $L_{\mathrm{sm}}\left({\mathit{x}}_i\right)=1$, the $L\left({\mathit{x}}_i\right)$ is $L\left({\mathit{x}}_i\right)=1+\beta$. If the $L_{\mathrm{sm}}\left({\mathit{x}}_i\right)>1$, the $L\left({\mathit{x}}_i\right)$ is $L\left({\mathit{x}}_i\right)=1+\beta L_{\mathrm{sm}}\left({\mathit{x}}_i\right)>1+\beta$, which denotes that the sample ${\mathit{x}}_i$ has less probability to be the exemplar (the real value of the similarity is the negative value of the similarity matrix). If the $\beta$ is 0, the $L\left({\mathit{x}}_i\right)$ degenerates to 1, which indicates that the LOF coefficient provides no influence on the original exemplar preference.

Finally, by combining Equations (17) and (20), $-S_{\mathrm{f}}\left({\mathit{x}}_i,{\mathit{x}}_k\right)$ is used as the similarity matrix of AP, and the clustering indices are obtained by the original affinity propagation.

3. Experiments

3.1. Hyperspectral Dataset

In our experiment, three HSI were used to test the performance of the proposed algorithm. The descriptions of the datasets are introduced as follows.

The Indian Pines (IP) dataset was gathered in 1992 by the AVIRIS sensor over the Indian Pines test site in northwest Indiana, United States. The size of the image is $145\times 145$ pixels and 220 spectral reflectance bands in the wavelength ranges from 0.40 to 2.50 $\mathsf{\mu }\mathrm{m}$. The spatial resolution is about 20 m. The available ground truth is designated into sixteen classes. The gray image and the reference land-cover map of the Indian Pines are shown in Figure 2. The land cover types with the number of samples are shown in

Table 1. Land cover type with the number of samples for the Indian Pines dataset.

Classes	Land Cover Type	Number of Samples
Class 1	Alfalfa	46
Class 2	Corn-Notill	1428
Class 3	Corn-Mintill	830
Class 4	Corn	237
Class 5	Grass-Pasture	483
Class 6	Grass-Trees	730
Class 7	Grass-Pasture-Mowed	28
Class 8	Hay-Windrowed	478
Class 9	Oats	20
Class 10	Soybean-Notill	972
Class 11	Soybean-Mintill	2455
Class 12	Soybean-Clean	593
Class 13	Wheat	205
Class 14	Woods	1265
Class 15	Buildings-Grass-Trees-Drives	386
Class 16	Stone-Steel-Towers	93

.

The Pavia University (PU) dataset was collected by the Reflective Optics System Imaging Spectrometer sensor during a flight campaign over the University of Pavia in Pavia, northern Italy. The size of the image was $610\times 340$ pixels. The spectral reflectance bands were 103 in the wavelength range from 0.43 to 0.86 $\mathsf{\mu }\mathrm{m}$. The geometric resolution was about 1.3 m. A part of the image with a size of $300\times 200$ pixels was selected in our experiment. The gray image and the reference land-cover map of the Pavia University are shown in Figure 3. The land cover types with the number of samples are shown in

Table 2. Land cover classes with the number of samples for the Pavia University dataset.

Classes	Land Cover Type	Number of Samples
Class 1	Asphalt	2578
Class 2	Meadows	5216
Class 3	Gravel	47
Class 4	Trees	1054
Class 5	Painted metal sheets	1345
Class 6	Bare Soil	868
Class 7	Bitumen	21
Class 8	Self-Blocking Bricks	1693
Class 9	Shadows	215

.

The WHU-Hi-HongHu (HH) dataset [61] was acquired in 2017 in Honghu City, Hubei Province, China, with a 17-mm focal length Headwall Nano-Hyperspec imaging sensor equipped on a DJI Matrice 600 Pro UAV platform. The size of the image was $940\times 475$ pixels. There were 270 bands from 0.40 to 1.00 $\mathsf{\mu }\mathrm{m}$. The spatial resolution was about 0.043 m. A part of the image with a size of $150\times 200$ pixels was selected in our experiment. The gray image and reference land-cover of the WHU-Hi-HongHu dataset are shown in Figure 4. The land cover types with the number of samples are shown in

Table 3. Land cover classes with the number of samples for Pavia University dataset.

Classes	Land Cover Type	Number of Samples
Class 1	Red roof	1981
Class 2	Road	1633
Class 3	Chinese cabbage	4902
Class 4	Cabbage	446
Class 5	Brassica parachinensis	6
Class 6	Brassica chinensis	367
Class 7	White radish	632
Class 8	Broad bean	1322
Class 9	Tree	4040

.

3.2. Experimental Setup

In order to evaluate the performance of the proposed method, three different types of HSIs (including two nature crops scenarios and one urban scenario) were conducted as part of the experiment. Consistent comparisons between CLAP based on the Euclidean distance (ED), the center-based unsupervised algorithms such as the K-means, K-mediods, Spectral clustering (SC) [62], AP as well as Gaussian mixture models (GMM) [63], DBSCAN [64], density peaks clustering (DPC) [65], self-organizing maps (Self-org) [66], competitive layers (CL) [67], HESSC [25], and GR-RSCNet [28] have been carried out. The estimations of the clustering performance provided by these algorithms are given by normalized mutual information (NMI) [68,69], F-measure [39], accuracy (ACC), and adjusted rand index (ARI), which are described as follows.

The mutual information was used to measure the information shared by two clusters and assess their similarity. Given dataset D with N samples and two clusterings of D, namely $U=\left\{U_1,U_2,\cdots ,U_R\right\}$ with R clusters, and $V=\left\{V_1,V_2,\cdots ,V_C\right\}$ with C clusters, the entropy of a cluster U can be defined as

$$H\left(U\right)=-{\displaystyle \sum }_{i=1}^{R}P\left(i\right)logP\left(i\right)$$

(21)

where $P\left(i\right)$ is the probability of an object falls into cluster $U_i$ and can be defined as $P\left(i\right)=\frac{\left|U_i\right|}{N}$. Similarly, the entropy of the clustering $V$ can be calculated as

$$H\left(V\right)=-{\displaystyle \sum }_{j=1}^{C}P\left(j\right)logP\left(j\right)$$

(22)

The mutual information between $U$ and $V$ can be described as

$$I\left(U,V\right)={\displaystyle \sum }_{i=1}^R{\displaystyle \sum }_{j=1}^{C}P\left(i,j\right)log\frac{P\left(i,j\right)}{P\left(i\right)P\left(j\right)}$$

(23)

where $P\left(i,j\right)=\frac{\left|U_i\cap V_j\right|}{N}$ denotes the probability that a point belongs to cluster $U_i$ in $U$ and cluster $V_j$ in $V$. The normalized version of the mutual information can be defined as

$$NMI\left(U,V\right)=\frac{I\left(U,V\right)}{\sqrt{H\left(U\right)H\left(V\right)}}$$

(24)

The F-measure is an agglomerative method to compare the overall set of clusters. Given cluster $U_i$ and $V_j$, the F-measure of these clusters is defined as

$$F\left(U_i,V_j\right)=\frac{2\ast Re\left(U_i,V_j\right)\ast Pr\left(U_i,V_j\right)}{Re\left(U_i,V_j\right)+Pr\left(U_i,V_j\right)}$$

(25)

where $Re\left(U_i,V_j\right)=\frac{\left|U_i\cap V_j\right|}{\left|V_j\right|}$ is the recall value, and $\mathrm{Pr}\left(U_i,V_j\right)=\frac{\left|U_i\cap V_j\right|}{\left|U_i\right|}$ is the precision value. The F-measure of the entire clustering solution is defined as the sum of the maximum F-measure of the individual cluster weighted by the cluster size, which can be expressed as

$$FM={\displaystyle \sum }_{i=1}^R\frac{\left|U_i\right|}{N}\underset{V_j\in V}{\max }F\left(U_i,V_j\right)$$

(26)

The cluster accuracy and adjusted Rand index are measurements used to evaluate clustering results. The cluster accuracy can be expressed as

$$ACC\left(U,V\right)=\frac{{{\displaystyle \sum }}_{i=1}^R\delta \left(U_i, map\left(V_i\right) \right)}{N}$$

(27)

where $map\left(·\right)$ indicates the best class of labels to reassign, and $\delta \left(·\right)$ is the indicator function and can be expressed as

$$\delta \left(x,y\right)=\left\{\begin{array}{c}1 if x=y\\ 0 otherwise\end{array}\right.$$

(28)

To obtain the adjusted Rand index, we first defined TP, TN, FP, FN. TP denotes the number of pairs of samples that are in the same cluster in U and are also in the same cluster in V; TN denotes the number of pairs of samples that are not in the same cluster in U and are not in the same cluster in V; FP denotes the number of pairs of samples that are not in the same cluster in U, but are in the same cluster in V; FN denotes the number of pairs of samples that are in the same cluster in U, but are not in the same cluster in V. We can define the Rand index as

$$RI\left(U,V\right)=\frac{TP+TN}{C_N^2}$$

(29)

where $C_N^2$ denotes the total number of pairs of samples that can be composed in the dataset. The ARI can be expressed as

$$ARI\left(U,V\right)=\frac{RI-E\left(RI\right)}{\max \left(RI\right)-E\left(RI\right)}$$

(30)

where $E\left(RI\right)$ is the expected index of the Rand index.

3.3. Experimental Results in Different HSIs

In this experiment, the proposed method and the competitors were conducted on three HSI datasets. For the proposed CLAP, we carried out the experiment with $\lambda =0.9$,$n=3$, $\alpha =0.5$, and $\beta =0.9$. The size of the neighborhood block was $25\times 25$ pixels ($k_w=25$). We decomposed the images using a complex version of a 1-scale, 16-orientation steerable pyramid decomposition, and $7\times 7$ window. The k-distance of LOF was set to 10. The exemplar preference of CLAP and traditional AP is in the range of $\left[-200,-mean\left(S_{pix}\right)\right]$, where $mean\left(S_{pix}\right)$ is the mean value of the pixel distance of the samples. In practice, the pixel distance is obtained by the ED and is normalized to $\left[0,1\right]$. As a result, the value of $mean\left(S_{pix}\right)$ is in the range $\left[0,1\right]$, which was less than 200. For the GR-RSCNet, the learning rate was set to 0.002, and the maximum training epoch was set to 20. The other parameters were set according to [28]. The parameters of other competitors were set according to the corresponding references.

From the comparison between the proposed CLAP and the other clustering methods, we observed that the proposed method was able to achieve the competitive performance in all considered datasets in Figure 5, Figure 6 and Figure 7. The number of mistaken clustering was obviously reduced in Figure 5l, Figure 6l, and Figure 7l. In Figure 5, the proposed method had a better clustering effect on Class 8, Class 13, and Class 14 in the clustering maps of the IP dataset than the competitive method, except for DPC and GR-RSCNet. However, we noticed that the DPC grouped the pixels from different land-covers into the same cluster. The clustering results were obviously unreasonable. Similar phenomena of the clustering results of DPC can be seen in Figure 6f and Figure 7f. Regarding GR-RSCNet, it is the state-of-the-art deep learning-based clustering method, which obtains significantly better clustering results than the traditional clustering algorithms. The same advantage of the proposed method can be seen in Class 2 in the clustering maps of the PU dataset in Figure 6, and on Class 1 and Class 7 in the clustering maps of the HH dataset in Figure 7. The experimental results demonstrate that our proposed method better serves the clustering task and can distinguish different types of land information well. Furthermore, as can be seen from the clustering maps, the proposed CLAP can effectively alleviate the salt-and-pepper phenomenon of the clustering result of the ground objects. Particularly for the HH dataset, the pepper phenomenon of Class 2 and Class 9 was obviously reduced in Figure 7l compared with the clustering results of the competitors except for GR-RSCNet. The main reason is that CLAP invites the spatial information to construct the similarity matrix of the pixels, while the competitors only use the spectral features to achieve the similarity.

In order to quantitatively compare the clustering performance, the NMI, F-measure, ACC, and ARI were used to evaluate the algorithms. We collected the average value of the 10 times clustering results of all the algorithms in the three HSI datasets. The results are shown in

Table 4. Comparison results of the NMI, F-measure, ACC, ARI, and running time of the algorithms in three HSI datasets.

Dataset	Method	NMI	FM	ACC	ARI	Time(s)
IP	K-means	0.4402	0.4083	0.3637	0.2218	2.24
	K-methods	0.4369	0.4064	0.3843	0.2186	11.9
	GMM	0.4338	0.4184	0.4417	0.2333	1.08
	DBSCAN	0.4202	0.4587	0.5274	0.2771	21.07
	SC	0.4431	0.4407	0.4410	0.2275	242.01
	DPC	0.4053	0.4988	0.8808	0.1976	43.99
	Self-org	0.4319	0.3879	0.3606	0.2077	42.61
	CL	0.4023	0.3757	0.3886	0.1848	564.22
	AP	0.4395	0.4418	0.4848	0.2638	356.8
	HESSC	0.4004	0.3609	0.3522	0.1917	286.74
	GR-RSCNet	0.5848	0.5368	0.5772	0.3378	3883.31
	CLAP	0.4525	0.4674	0.5334	0.3237	661.51
PU	K-means	0.6901	0.7219	0.7012	0.6060	0.41
	K-methods	0.7078	0.7770	0.7463	0.6394	5.12
	GMM	0.6305	0.6466	0.6923	0.5596	0.78
	DBSCAN	0.6523	0.7034	0.7388	0.5490	16.3
	SC	0.4815	0.6779	0.7519	0.3646	490.49
	DPC	0.4373	0.6770	0.8131	0.2973	43.66
	Self-org	0.6469	0.6530	0.6695	0.5081	42.68
	CL	0.5898	0.5691	0.5146	0.3597	593.55
	AP	0.7066	0.7600	0.7592	0.6639	634.7
	HESSC	0.5228	0.5648	0.6087	0.3871	430.07
	GR-RSCNet	0.8623	0.8183	0.8030	0.7041	2430.22
	CLAP	0.6832	0.7807	0.7608	0.7540	936.55
HH	K-means	0.5033	0.5533	0.5165	0.3038	2.77
	K-methods	0.5097	0.5602	0.5308	0.3148	18.96
	GMM	0.5930	0.6486	0.6588	0.4043	2.22
	DBSCAN	0.4959	0.5478	0.6279	0.3003	54.36
	SC	0.4815	0.6779	0.7519	0.3646	490.49
	DPC	0.4823	0.5894	0.8980	0.3258	89.65
	Self-org	0.5030	0.5532	0.5167	0.3035	42.272
	CL	0.4843	0.5222	0.4795	0.2743	625.25
	AP	0.4811	0.5698	0.5689	0.3073	623.79
	HESSC	0.4651	0.4962	0.4940	0.2692	334.21
	GR-RSCNet	0.8424	0.7920	0.7693	0.6970	3759.08
	CLAP	0.5460	0.5898	0.5715	0.3488	947.07

.

In Table 4, we can see that CLAP provided competitive clustering results in all HSI datasets. For the IP dataset, the NMI of CLAP was 0.4525, which was higher than that of the competitors, except for GR-RSCNet. Similarly, the ARI of CLAP (0.3237) was the second highest among the competitors. We noted that the FM and ACC of the DPC was higher than that of CLAP. However, it can be seen from Figure 5f that the clustering result of DPC was inappropriate compared to the real land-cover. Similar conclusions can be obtained for the PU dataset and HH dataset. GMM provided the second-best clustering results in the HH dataset. The CLAP obtained the third-best clustering results in the HH dataset and indicates that the HH dataset is more suitable to be clustered by GMM. The GR-RSCNet obtained the best clustering results among the competitors and showed that the potential of the deep learning-based clustering methods was significantly greater than that of the traditional clustering methods. More specifically, we focused on the comparison of the proposed CLAP with the AP. It can be seen that CLAP provided higher clustering results on all datasets than that of AP and shows that CLAP can effectively improve the performance of the original AP. In addition, we could see that CLAP required more running time than that of the AP. The additional running time was used to calculate the CW-SSIM and LOF. The GR-RSCNet provided the longest running time among the competitors.

3.4. The Optimization Strategy of $\alpha$, $\beta$, and $k_w$

In this section, the settings of parameters, $\alpha$, $\beta$, and $k_w$ in our CLAP are discussed. To achieve the best clustering performance, these parameters were tuned according to the results of our proposed algorithm running on the HSI datasets. Concerning different values of $\alpha$, which varied from 0 to 1, the clustering precision of the CLAP on the three datasets are shown in Figure 8a–c. It should be noted that $\alpha$ is the weight of the spatial structure similarity of ground objects according to Equation (17). Specifically, when $\alpha$ is set to 0, the proposed CLAP is simplified to the original AP with only the LOF weighted term. Figure 8a shows that the best clustering result was obtained when the preference selection of α was around 0.5 on the IP dataset. In Figure 8b,c, it can be observed that on the PU dataset, it was around 0.3, and on the HH dataset, it was 0.4, respectively. Furthermore, it should be clearly observed that due to adding the weight term $\alpha$, the clustering performance of the proposed CLAP was better than original AP algorithm ($\alpha =0$). At the same time, through comparative analysis, it can be found that $\alpha$ taking a large value (e.g., $\alpha$ is set to greater than 0.8.) may result in that the modified fusion distance (in Equation (17)) depends more on spectral similarity, which in turn reduces the clustering accuracy of the algorithm. This may be because the over-weighted spatial similarity exceeded the threshold of the spectral similarity, introducing too much spatial information, which affects the clustering precision.

Concerning different values of $\beta$, which varies from 0 to 1, the clustering precision of the CLAP on the three datasets are shown in Figure 9a–c. According to Equation (19), $\beta$ is the weight of $L_{sm}$ in the LOF coefficient, which is related to estimate suitable exemplar preference. Figure 9a shows that the best clustering result was obtained when the preference selection of $\beta$ was around 0.9 on the IP dataset. From Figure 9b,c it can be observed that it was around 0.8 on both the PU dataset and HH dataset. The experimental results show that a larger $\beta$ is better for estimating suitable exemplar preference.

Concerning different values of $k_w$, which is the size of the spatial neighbourhood block and varies from 13 to 31, the clustering precision of CLAP on the three datasets are shown in Figure 10a–c. Figure 10a shows that on the IP dataset, when the $k_w$ is set to about 25 (i.e., the size of block is $25\times 25$ pixels), the best clustering result was obtained. Similarly, in Figure 10b,c, it can be observed that on the PU dataset, it was $25\times 25$ and on the HH dataset, it was $29\times 29$, respectively. The experimental results show that its value of $k_w$ is too small (eg. $15\times 15$), which can result in the poor clustering performance. It may be because too small a block may result in missing useful structural information, which in turn will reduce the clustering performance.

4. Discussion

In this paper, the structural similarity index and local outlier factor were introduced to improve the original AP clustering. Comparisons were conducted on three HSI datasets. The visual and statistical results are shown in Figure 5, Figure 6 and Figure 7 and Table 4. The influence of the parameters is discussed in Section 3.4.

From the clustering results of CLAP and AP, we can see that CLAP provided better results than AP on all three HSI datasets. It is understandable that we collected more information from the HSI dataset in CLAP. The CW-SSIM can be denoted as spatial information. The LOF can be denoted as spectral information and indicates that the extraction of the spatial-spectral information of the HSI dataset can effectively improve the performance of the clustering algorithms. This practice gives us an idea of how to improve the algorithms for processing the HSI dataset. Although CLAP requires a longer running time than that of AP, the improved strategy for calculating the CW-SSIM distance is still in effect, where the running time was significantly reduced compared to the direct usage of CW-SSIM between a pair of spatial neighborhood blocks. For example, the running time of the original CW-SSIM distance strategy was more than five hours on the IP dataset, and that of the improved version was about 290 s.

Furthermore, broader comparisons were conducted in the experiment. From the comparison results, we can see that the proposed CLAP provided competitive clustering performance, which was in the top three in all indicators on three HSI datasets. The deep learning-based clustering method provided the best clustering result among the competitors, which showed great potential to address the issue of the HSI clustering. However, the deep learning-based clustering method provided the highest running time (the running time was obtained on the GPU platform of Tesla V100 16G) than that of the traditional clustering methods (the running time was obtained on the CPU platform of Intel Core i5-6200U). There is still interesting work to improve the efficiency of the deep learning-based clustering method.

Meanwhile, there are still some drawbacks to the proposed CLAP. First, the proposed CLAP has many parameters to be modulated such as $\alpha ,\beta ,k_w$. The optimal values of these parameters vary according to the dataset. To simplify the optimization, we provided a set of initial parameters ($\alpha =0.5,\beta =0.9,k_w=25$) designed to help optimize the parameters. Experiments showed that satisfactory results can be obtained by selecting parameters around the initial parameters. Second, CW-SSIM was used to extract the structure-based spatial similarity of the HSI dataset. Experiments showed that it had poor performance of spatial similarity, which needs to precisely control the weighting of the spatial similarity. Finally, CLAP requires a global similarity matrix to transfer information between samples. The global similarity matrix needs large storage space to store a large scale HSI dataset.

5. Conclusions

In this paper, a modified AP based on CW-SSIM and LOF was proposed. The CW-SSIM was used to extract the structure-based spatial similarity of the HSI dataset, which was combined with the pixel-based spectral similarity to generate the final similarity matrix of AP. The LOF was applied to measure the smoothness of an object and used to revise the exemplar preference of AP. Meanwhile, we simplified the calculation of the spatial similarity to reduce the computational complexity. The modified similarity matrix was obtained by the pixel-based spectral similarity, the structure-based spatial similarity, and the revised exemplar preference. Finally, the modified similarity matrix was applied to AP and the clustering index was obtained.

To evaluate the effectiveness of the proposed CLAP, comparisons were carried out between CLAP, AP, K-means, K-methods, GMM, DBSCAN, SC, DPC, Self-org, CL, HESSC, and GR-RSCNet on three different types of HSI datasets. The experimental results showed that the proposed CLAP could distinguish different types of land covers well and outperformed its competitors. Meanwhile, the optimization strategy of the main parameters of CLAP was also discussed. From the clustering results, we can see that the weight of spatial similarity should be tuned carefully as too large values may reduce the clustering precision of the algorithm. The weight of the LOF coefficient and the size of the spatial neighborhood block also have an impact on the clustering result. The comparison and drawbacks are discussed in the Discussion section. We can see that CLAP outperformed its competitors, but still suffers from some issues such as difficulty in selecting parameters, unstable performance of the CW-SSIM, and high storage requirements. Further work will focus on the improvement in the scheme to efficiently extract the spatial-spectral information of the HSI dataset in combination with deep learning-based HSI clustering algorithms.