Quadratic Clustering-Based Simplex Volume Maximization for Hyperspectral Endmember Extraction

Abstract

The existence of intra-class spectral variability caused by differential scene components and illumination conditions limits the improvement of endmember extraction accuracy, as most endmember extraction algorithms directly find pixels in the hyperspectral image as endmembers. This paper develops a quadratic clustering-based simplex volume maximization (CSVM) approach to effectively alleviate spectral variability and extract endmembers. CSVM first adopts spatial clustering based on simple linear iterative clustering to obtain a set of homogeneous partitions and uses spectral purity analysis to choose pure pixels. The average of the chosen pixels in each partition is taken as a representative endmember, which reduces the effect of local-scope spectral variability. Then an improved spectral clustering based on k-means is implemented to merge homologous representative endmembers to further reduce the effect of large-scope spectral variability, and final endmember collection is determined by the simplex with maximum volume. Experimental results show that CSVM reduces the average spectral angle distance on Samson, Jasper Ridge and Cuprite datasets to below 0.02, 0.06 and 0.09, respectively, provides the root mean square errors of abundance maps on Samson and Jasper Ridge datasets below 0.25 and 0.10, and exhibits good noise robustness. By contrast, CSVM provides better results than other state-of-the-art algorithms.

1. Introduction

Due to the limitation of spatial resolution and the complexity of ground object distribution, mixed pixels exist in hyperspectral images (HSIs) and reduce the accuracy of traditional pixel-level applications [1]. Therefore, it is necessary to perform hyperspectral unmixing for further analysis, which includes extracting pure pixels (endmembers) and determining the fractions of each endmember in pixels (abundance). The former is called endmember extraction, and the latter is called abundance estimation. Hyperspectral unmixing has been used in many fields to improve the related performance, such as subpixel target detection, precise land cover mapping and Earth change detection [2,3,4,5,6,7].

The mixing model is the basis for the research of hyperspectral unmixing, containing the linear spectral mixing model (LMM) and nonlinear spectral mixing model (NLMM) [8,9]. LMM assumes the photons arriving at the sensor interact with one ground object, so each mixed pixel is described as a linear combination of endmembers weighted by the corresponding abundance. As a matter of fact, the photons will go through multiple reflections and refractions between various ground objects, leading to NLMM. It has no fixed expression and needs to be determined for specific land cover class. Hence, LMM is utilized in most endmember extraction algorithms (EEAs) due to the simple structure and good performance, and it is also used in our work.

EEAs based on convex geometry are widely used in linear spectral unmixing. The pure pixel index (PPI) projects all pixels onto a set of vectors, and pixels that fall at the endpoints of these vectors the most are selected as endmembers [10]. Based on PPI, vertex component analysis (VCA) uses the orthogonal direction of the subspace spanned by the determined endmembers as projection axis, and pixels with the extreme value is taken as a new endmember [11]. N-FINDR works by finding the simplex with maximum volume in the space of the HSI, and pixels that constitute this simplex are regarded as endmembers [12]. As a variant of N-FINDR, the simplex growing algorithm (SGA) reduces the computational cost by including a process of growing simplexes one vertex at a time [13]. The alternating volume maximization (AVMAX) improves N-FINDR, using an alternating optimization strategy to pin down some convergence characteristics [14]. To enhance the noise robustness, the minimum volume simplex algorithm (MVSA) fits the minimum simplex volume to the data and reduces the effect of noise and outliers by imposing a hinge type soft constraint on the abundance [15]. The fast subspace-based module (FSPM) detects convex hull vertices for component pairs obtained by singular value decomposition and proposes a local outlier score measurement to remove outliers [16]. The statistics-based EEAs decompose the data into the product of endmember matrix and abundance matrix simultaneously, and iteratively solves the problem by adding prior constrained information. This type of method handles HSIs with a high mixing degree well but requires a lot of computations. The nonnegative matrix factorization (NMF) is a commonly used method. Based on NMF, several improved approaches have emerged by adding different constraints, such as minimum volume constrained NMF (MVC-NMF), ${\mathrm{L}}_{1/2}$-NMF and the robust collaborative NMF (Co-NMF) [17,18,19].

In HSIs, pure pixels are more likely to exist in homogeneous areas, and mixed pixels are more likely to be present in transition areas containing two or three features. The above two types of methods ignore such spatial properties and identify endmembers only by spectral features, resulting in limited endmember extraction performance. Therefore, EEAs that fuse spatial and spectral features have appeared, which improve the accuracy of endmember extraction to a certain extent. Automatic morphological endmember extraction (AMEE), the earliest algorithm to fuse spatial and spectral properties, enables unsupervised endmember extraction by performing multidimensional morphological dilation and erosion over an ever-increasing spatial neighborhood [20]. The entropy-based convex set optimization (ECSO), developed in recent years, exploits entropy as spatial information to choose two-band pairs and calculate the convex set for two-band data to determine endmembers [21].

As clustering can effectively utilize spatial and spectral information, it has been widely used in endmember extraction, further improving the performance of hyperspectral unmixing. For example, clustering and over-segmentation-based preprocessing (COPP) combines top–down over-segmentation (TDOS) with spectral clustering to generate homogeneous partitions, and the most spectrally pure average vectors are sent to spectral-based EEAs [22]. The regional clustering-based spatial preprocessing (RCSPP) performs regional clustering to obtain a set of clustered partitions that exhibit spectral similarity, then a subset of candidate pixels with high spectral purity is fed to endmember detection methods [23]. As a recently proposed method, spatial energy prior constrained maximum simplex volume (SENMAV) utilizes spatial energy prior obtained by spectral clustering as a regularization term of the maximum simplex volume model to constrain the selection of endmembers [24]. The spatial potential energy weighting endmember extraction (SPEW) adopts spectral angular distance and space potential energy derived from spectral clustering to weight pixels, and the weighted pixels that compose the maximum volume simplex are regarded as simplex [25]. K-medoids-based endmember extraction (KMEE) applies convex geometry for the selected two bands and uses K-medoids clustering to remove extra points, aiming to improve the noise robustness [26].

Spectral variability exists in HSIs due to the different scene components and illumination conditions, which blurs the identification of different types of ground objects and brings inaccurate endmember representation. Large-scope spectral variability refers to the phenomenon that homogeneous ground objects at different positions in an image show different spectral signatures. Local-scope spectral variability refers to the spectral difference exhibited by homogeneous ground objects in adjacent areas, which is small since the scene conditions may not change much in a local area. As most of the current methods directly find pixels in the image as endmembers, the influence of spectral variability cannot be avoided. Spatial purity-based endmember extraction (SPEE) attempts to address the problem by using the average of pure spatial neighborhoods to mitigate local-scope spectral variability and the merge of spatially related candidates to mitigate large-scope spectral variability [27].

In this paper, a novel quadratic clustering-based simplex volume maximization (CSVM) method is proposed to efficiently extract endmember, which exploits the advantages of clustering to make full use of spatial and spectral information and better deal with spectral variability. CVSM first adopts spatial clustering to segment HSI into homogeneous partitions, and the average of pure pixels obtained by spectral purity analysis in each partition is taken as representative endmember, which reduces the effect of local-scope spectral variability. Then CVSM performs spectral clustering to merge spectrally related representative endmembers to further alleviate large-scope spectral variability, and the final endmembers are determined by simplex with maximum volume. The CSVM shows its novelty in three ways. First, a local clustering criterion is proposed for spatial clustering based on simple linear iterative clustering (SLIC) to acquire a set of partitions with good spatial homogeneity, which combines spectral angle distance (SAD) and Euclidean distance (${\mathrm{ED}}_{\mathrm{spe}}$) to constraint spectral measurement. Second, spectral clustering adopts an improved weighted k-means algorithm to adjust the clustering performance, which takes both spectral curve shape and distance into consideration to alleviate the large-scope spectral variability. Third, a set of candidates with a small number of pixels is obtained after quadratic clustering, which reduces the complexity of the simplex-based algorithm.

The major contributions of this paper are threefold. First, a quadratic clustering-based endmember extraction method is introduced to better deal with spectral variability, as it is one of the current limitations to improve the hyperspectral unmixing performance. Second, parameter analysis is conducted to choose the optimal value of each parameter involved in CSVM, thus enhancing the applicability of CSVM to different datasets. Finally, experiments using real hyperspectral prove that our algorithm has great improvements in endmember extraction, abundance estimation and anti-noise performance, which also indicates that the spectral variability cannot be ignored in hyperspectral unmixing.

2. Proposed Methods

The proposed method, CSVM, is implemented from spatial and spectral dimension, as shown in Figure 1. The processing in spatial dimension contributes to handle local-scope spectral variability, while the processing in spectral dimension aims to deal with large-scope spectral variability and makes the final decision on endmember extraction. CSVM will be introduced in four steps, which are spatial clustering, spectral purity analysis, spectral clustering and simplex volume maximization.

2.1. Spatial Clustering

Spatial clustering can be achieved by superpixel segmentation algorithms. Here, the simple linear iterative clustering (SLIC) algorithm commonly used in two-dimensional image processing is enhanced for HSIs, as it has significant advantages in terms of calculation speed, memory efficiency and segmentation performance [28,29]. The enhancement mainly focuses on the adjustment to the clustering criterion, making it better meet the need of regional homogeneity in partitions. The details of spatial clustering are as follows.

The HSI is first divided into blocks with the size of $\mathrm{s} \times \mathrm{s}$ as the initial clustering state. Take the center of each block as the cluster center. Considering that the centers located at the edge in the image may affect the clustering performance, the principal component analysis (PCA) is performed, and the projection of the image to the first principal component is chosen to calculate the gradient. The pixel with the lowest gradient in each block is regarded as new cluster center. In addition, PCA has good denoising effect, and the first principal component contains most of the information in the HSI.

Since good spatial homogeneity makes the representative endmembers more accurate, it is our primary consideration in determining the clustering principle. In some studies, clustering strategies have been developed to produce superpixels with regular sizes and shapes but poor spectral similarity, which is not suitable for us [30]. In this work, the combination of ${\mathrm{ED}}_{\mathrm{spe}}$ and $\mathrm{SAD}$ provides constraints on spectral similarity as spectral measurement, and spatial Euclidean distance (${\mathrm{ED}}_{\mathrm{spa}}$) is utilized as the spatial measurement. They are formulated as follows:

$${\mathrm{ED}}_{\mathrm{spa}}=\sqrt{{\left({\mathrm{x}}_{\mathrm{i}}-{\widehat{ \mathrm{x}}}_{\mathrm{i}}\right)}^2+{\left({\mathrm{y}}_{\mathrm{i}}-{\widehat{ \mathrm{y}}}_{\mathrm{i}}\right)}^2}/\mathrm{L},$$

(1)

$${\mathrm{ED}}_{\mathrm{spe}}=\sqrt{\Vert {\mathbf{v}}_{\mathrm{i} }-{ \mathbf{c}}_{\mathrm{i}}\Vert {}_2^2/\mathrm{b}},$$

(2)

$${\mathrm{SAD}=\cos }^{-1}\left(\frac{{<\mathbf{v}}_{\mathrm{i}}{,\mathbf{c}}_{\mathrm{i}}>}{\Vert {\mathbf{v}}_{\mathrm{i}}\Vert ·\Vert {\mathbf{c}}_{\mathrm{i}}\Vert }\right),$$

(3)

where ${(\mathrm{x}}_{\mathrm{i}},{\mathrm{y}}_{\mathrm{i}})$ refers to the location of a given pixel in the image, $({\widehat{ \mathrm{x}}}_{\mathrm{i}},{\widehat{ \mathrm{y}}}_{\mathrm{i}})$ is the center location of the partition that contains the pixel, $\mathrm{L}$ is the length of the diagonal for the search scope used to normalize the ${\mathrm{ED}}_{\mathrm{spa}}$, ${\mathbf{v}}_{\mathrm{i}}$ and ${\mathbf{c}}_{\mathrm{i}}$ are the spectra of the pixel and the cluster center respectively, and $\mathrm{b}$ is the number of bands in the HSI.

Thereby, the spatial measurement ${\mathrm{D}}_{\mathrm{spa}}$ and the spectral measurement ${\mathrm{D}}_{\mathrm{spe}}$ are defined as follows:

$${\mathrm{D}}_{\mathrm{spa}}{=\mathrm{ED}}_{\mathrm{spa}},$$

(4)

$${\mathrm{D}}_{\mathrm{spe}}=\frac{\left({\mathrm{ED}}_{\mathrm{spe}}+\mathrm{SAD}\right)}{2}.$$

(5)

The clustering criterion $\mathrm{D}$ is defined as follows:

$${\mathrm{D}=\mathsf{\lambda }}_1{ \times \mathrm{D}}_{\mathrm{spa}}+\left(1 -{ \mathsf{\lambda }}_1\right){ \times \mathrm{D}}_{\mathrm{spe}},$$

(6)

where ${\mathsf{\lambda }}_1$ is a weighting factor to balance ${\mathrm{D}}_{\mathrm{spa}}$and ${\mathrm{D}}_{\mathrm{spe}}$.

After initialization, spatial clustering can be iteratively conducted. In each iteration, calculate $\mathrm{D}$ for pixels within the search scope of $2 \mathrm{s} \times 2 \mathrm{s}$ for each cluster center. Since a pixel may be searched for more than one time and obtain multiple $\mathrm{Ds}$, assign it to the cluster center corresponding to the smallest $\mathrm{D}$. Then update the cluster centers and their corresponding spectra to enter the next iteration. Pixels belonging to the same cluster center form a partition. The spatial clustering ends when all partitions no longer change. It should be noted that the designed search scope greatly reduces the computational complexity compared to global searching and makes it possible to accelerate our algorithm using parallel computation [31].

2.2. Spectral Purity Analysis

Spatial clustering obtains a set of homogeneous partitions, but the existence of mixed pixels in each partition cannot be avoided, which reduces the accuracy of representative endmembers. It is necessary to perform spectral purity analysis to choose pure pixels and remove the mixed pixels.

In the HSI, if all pixels are projected onto the same vector, endmembers will be projected to the endpoints of the vector, that is, endmembers will obtain maximum or minimum projection values. Given that each partition is considered to mainly contain one feature, pixels with higher projection values are more likely to be pure pixels. Based on that, PCA is performed on each partition, and a set of principal components decomposed by PCA is obtained denoted by ${\mathbf{U}=\{\mathbf{u}}_{\mathrm{i}}{\}}_{\mathrm{i}=1}^{\mathrm{b}}$. The first principal component ${\mathbf{u}}_1$ is chosen as the projection vector, as it captures most of the information of the hyperspectral data. Then all pixels in each partition are projected on it as follows:

$${\mathbf{x}=\mathbf{u}}_1{}^{\mathrm{T}}\mathbf{S},$$

(7)

where ${\mathbf{S}=(\mathbf{s}}_1{,\mathbf{s}}_2{,\dots ,\mathbf{s}}_{\mathrm{N}})\in {ℝ}^{\mathrm{b}\times \mathrm{N}}$ is a partition containing $\mathrm{N}$ pixels, and ${\mathbf{x}=(\mathrm{x}}_1{,\mathrm{x}}_2{,\dots ,\mathrm{x}}_{\mathrm{N}})\in {ℝ}^{1\times \mathrm{N}}$ is a set of projection values of this partition.

To find pixels with high spectral purity, sort $\mathbf{x}$ in descending order and obtain new vector $\widehat{\mathbf{x}}$. Choose pixels in order according to certain proportion Pt from $\widehat{\mathbf{x}}$, and take the average of the chosen pixels as a representative endmember for each partition, which alleviates the local-scope spectral variability and random noise.

2.3. Spectral Clustering

Since the wide area covered by one class, such as ocean and forest, will be segmented into several partitions due to the constraint of the search scope, representative endmembers from these partitions exhibit the same feature. The spectral clustering plays a role in merging the representative endmembers separated. On the one hand, it is of great significance to eliminate the effect of large-scope spectral variability. On the other hand, it can reduce the number of candidate endmembers, improving the computational efficiency of the subsequent processing.

The spectral clustering method is based on the k-means clustering algorithm [32]. The clustering criterion adopts a weighted spectral measurement strategy, which takes both spectra shape and distance into consideration. It is defined as follows:

$${\mathrm{D}=\mathsf{\lambda }}_2{ \times \mathrm{ED}}_{\mathrm{spe}}+\left(1 -{ \mathsf{\lambda }}_2\right) \times \mathrm{SAD},$$

(8)

where ${\mathsf{\lambda }}_2$ is a balance constant.

In spectral clustering, initialize $\mathrm{k}$ cluster centers first. Then calculate $\mathrm{D}$ between each representative endmember and the cluster centers. Each endmember is assigned to the center with the smallest $\mathrm{D}$. Next, update the cluster centers according to the classification results. Perform the above operations iteratively. Representative endmembers belonging to the same cluster center form a class. The spatial clustering ends when all classes no longer change. Finally, the cluster centers are regarded as candidate endmembers, which further suppresses the effect of large-scope spectral variability.

As the final endmembers will be chosen from candidate endmembers, that is, from the cluster centers, the performance of spectral clustering is particularly key, as is the determination of the two parameters involved.

2.4. Simplex Volume Maximization

The simplex composed of endmembers has the maximum volume. Based on that, the proposed method makes the final decision on endmembers.

Suppose that the set of candidate endmembers obtained by spectral clustering is ${\mathbf{Y}=(\mathbf{y}}_1{,\mathbf{y}}_2{,\dots ,\mathbf{y}}_{\mathrm{M}})\in {ℝ}^{\mathrm{b}\times \mathrm{M}}$. First, it is necessary to perform PCA on $\mathbf{Y}$to reduce the dimension to $\mathrm{p} - 1$ and obtain a new set denoted by $\widehat{\mathbf{Y}}=\left({\widehat{\mathbf{y}}}_{1^{\prime }},{\widehat{\mathbf{y}}}_{2^{\prime }}\dots ,{\widehat{\mathbf{y}}}_{\mathrm{M}}\right)\in {\mathbb{R}}^{(\mathrm{p}-1)\times \mathrm{M}}$. $\mathrm{P}$ represents the number of endmembers. The simplex volume is calculated as follows:

$$\mathrm{V}\left({\widehat{\mathbf{y}}}_{{\mathrm{k}}_1},{\widehat{\mathbf{y}}}_{{\mathrm{k}}_2},\dots ,{\widehat{\mathbf{y}}}_{{\mathbf{k}}_{\mathrm{p}}}\right)=\frac{abs\left(\left|\begin{array}{cccc}1& 1& \dots & 1\\ {\widehat{\mathbf{y}}}_{{\mathrm{k}}_1}& {\widehat{\mathbf{y}}}_{{\mathrm{k}}_2}& \dots & {\widehat{\mathbf{y}}}_{{\mathrm{k}}_{\mathrm{p}}}\end{array}\right|\right)}{(\mathrm{p}-1)!}.$$

(9)

Iteratively select $\mathrm{p}$ vectors from $\widehat{\mathbf{Y}}$ and calculate the corresponding $\mathrm{V}$. The $\mathrm{p}$ vectors with the largest $\mathrm{V}$ are selected as the final endmembers. It can be formulated as follows:

$${\mathrm{V}}_{\_max}=\underset{{\widehat{\mathbf{y}}}_{{\mathrm{k}}_1},{\widehat{\mathbf{y}}}_{{\mathrm{k}}_2},\dots ,{\widehat{\mathbf{y}}}_{{\mathrm{k}}_{\mathrm{p}}}}{argmax}\mathrm{V}\left({\widehat{\mathbf{y}}}_{{\mathrm{k}}_1},{\widehat{\mathbf{y}}}_{{\mathrm{k}}_2},\dots ,{\widehat{\mathbf{y}}}_{{\mathrm{k}}_{\mathrm{p}}}\right).$$

(10)

3. Results and Discussion

In this section, parameter analysis is first carried out to choose the optimal value for each parameter in CSVM and improve the applicability of CSVM for future use. Then four experiments are performed on three datasets (the Samson dataset, Jasper Ridge dataset, and Cuprite dataset) to evaluate the performance of the proposed method. The comparison algorithms include VCA, SGA, AVMAX, MVSA, AMEE, ESCO, KMEE, FSPM, SENMAV, SPEW. Among them, VCA, SGA and AVMAX are convex geometry-based methods commonly used for comparison; AMEE and ESCO are typical methods that integrate spatial and spectral information; MVSA, KMEE and FSPM are noise-robust algorithms, and the result of FSPM is sent to VCA to determine the final endmembers, as it is a preprocessing module; SENMAV and SPEW combine clustering with a simplex-based algorithm, which is similar to our approach. Since hyperspectral unmixing involves endmember extraction and abundance estimation, the first two experiments aim to compare the performance of all test algorithms on endmember extraction and abundance estimation, respectively. The third experiment verifies the noise robustness of all algorithms by setting the signal-to-noise (SNR) level of the datasets to 15 dB, 20 dB, 25 dB, 30 dB, 35 dB and 40 dB. In the fourth experiment, the processing time of all methods are compared, and the acceleration possibility of the proposed method is analyzed. To ensure the reliability of the experimental results, each experiment is conducted five times and the average values are taken in this paper.

3.1. Input Data

3.1.1. Samson Dataset

Samson contains 156 bands covering the wavelength range from 401 nm to 889 nm with the spectral resolution of 3.13 nm. It has no blank channels or badly noised channels. To reduce the computational cost, the Samson dataset used in our experiment is partially cropped from the original image starting with the (252,332) pixel, which has the size of 95 × 95 and three endmembers corresponding to soil, tree, and water. The pseudo-color image and the spectral reflectance of endmembers are shown in Figure 2 [33].

3.1.2. Jasper Ridge Dataset

Jasper Ridge has 224 bands ranging from 380 nm to 2500 nm with the spectral resolution of 9.46 nm. Due to the dense water vapor and atmosphere effect, bands 1–3, 108–112, 154–166 and 220–224 are removed, and 198 bands are reserved for experiments. Additionally, a sub-image of 100 × 100 pixels starting with the (105,269) pixel from the original image is used, as shown in Figure 3. There are four endmembers in this data: road, soil, water and tree.

3.1.3. Cuprite Dataset

Cuprite is one of the most widely used hyperspectral data for hyperspectral unmixing, which contains 224 bands ranging from 370 nm to 2480 nm. The noised bands (1–2 and 221–224) and the water absorption bands (104–113 and 148–167) are removed, and 188 bands are retained. A sub-image of 250 × 190 pixels is utilized in our experiments, including 14 minerals. To avoid the minor difference between the variants of the same mineral, 12 minerals are finally considered as endmembers, which are alunite, andradite, buddingtonite, dumortierite, kaolinite1, kaolinite2, muscovite, montmorillonite, nontronite, pyrope, sphene and chalcedony as shown in Figure 4.

3.2. Parameter Analysis

The input parameters for CSVM include $\mathrm{p}$, $\mathrm{s}$, ${\mathsf{\lambda }}_1$, $\mathrm{Pt}$, ${\mathsf{\lambda }}_2$ and $\mathrm{k}$. The number of endmembers is denoted by $\mathrm{p}$, which can be obtained by endmember number estimation algorithms. The grid step s, the weighting factor ${\mathsf{\lambda }}_1$ and the proportion for representative endmember extraction $\mathrm{Pt}$ will be set through the following spatial clustering analysis. The weighting factor ${\mathsf{\lambda }}_2$ and the number of clusters $\mathrm{k}$ will be determined in the following spectral clustering analysis.

Indexes are first introduced to provide a quantitative assessment for the clustering performance, including $\mathrm{Scatt}$, $\mathrm{AvgPI}$ and $\mathrm{SvdPI}$ corresponding to the following definitions.

$$\mathrm{Scatt}=\frac{\Vert \mathsf{\sigma }\left({\mathbf{v}}_{\mathrm{i}}\right)\Vert }{\Vert \mathsf{\sigma }\left(\mathbf{X}\right)\Vert },$$

(11)

where $\mathsf{\sigma }\left(\mathbf{X}\right)$ is the variance of HSI and ${\mathsf{\sigma }(\mathrm{v}}_{\mathrm{i}})$ is the variance of the ith partition.

$$AvgPI=\frac{{\sum }_{\mathrm{i}=1}^{\mathrm{m}}dist\left({\mathbf{x}}_{\mathrm{i}},\overline{\mathbf{x}}\right)}{\mathrm{m}},$$

(12)

where $\mathrm{dist}$ is the correlation coefficient between ${\mathbf{x}}_{\mathrm{i}}$ and the average vector $\overline{\mathbf{x}}$ in the ith partition, and $\mathrm{m}$ is the number of pixels contained in this partition.

$$\mathrm{SvdPI}=\frac{{\mathrm{L}}_1}{{{\displaystyle \sum }}_{\mathrm{i}=1}^{\mathrm{m}}{\mathrm{L}}_{\mathrm{j}}},$$

(13)

where ${\mathrm{L}}_{\mathrm{j}}$ is the jth singular value after singular value decomposition.

The smaller the value of Scatt and the larger the value of $\mathrm{AvgPI}$ and $\mathrm{SvdPI}$, the better the clustering results.

3.2.1. Spatial Clustering Analysis

Spatial clustering involves the grid step $\mathrm{s}$ and the weighting factor for spatial measurement ${\mathsf{\lambda }}_1$. Two experiments are designed to determine the optimal values of them. To determine s, set ${\mathsf{\lambda }}_1$ to 0.9, and calculate $\mathrm{Scatt}$, $\mathrm{AvgPI}$ and $\mathrm{SvdPI}$ on the input datasets clustered with $\mathrm{s}$ from 5 to 30. The results are shown in Figure 5. To determine ${\mathsf{\lambda }}_1$, set $\mathrm{s}$ to 6, and $\mathrm{Scatt}$, $\mathrm{AvgPI}$ and $\mathrm{SvdPI}$ are calculated on the input datasets clustered with ${\mathsf{\lambda }}_1$ from 0 to 1 at an interval of 0.05. The results are shown in Figure 6.

From Figure 5, the indexes fluctuate significantly with the increase in $\mathrm{s}$, and from Figure 6, the indexes keep stable with the increase in ${\mathsf{\lambda }}_1$. It shows that $\mathrm{s}$ has a greater impact on the clustering performance than ${\mathsf{\lambda }}_1$.

Of all the datasets, Cuprite is in sharp contrast with Samson and Jasper Ridge. The indexes on Cuprite turn out to be the most stable. Especially in Figure 5, the indexes have a relatively slight change, with $\mathrm{s}$ rising from 5 to 30. This has relations with the spectral contrast of endmembers in datasets. As shown in Figure 4, Cuprite contains endmembers with minor difference, while endmembers in Samson and Jasper Ridge have high spectral contrast.

With the rise in $\mathrm{s}$, both $\mathrm{AvgPI}$ and $\mathrm{SvdPI}$ trend down and $\mathrm{Scatt}$ trends up. It can be inferred that a large $\mathrm{s}$ will lead to high dispersion degree and low spectral correlation as each partition is expanded. However, a small $\mathrm{s}$ will increase the computational cost. As is presented in Figure 5, the indexes keep steady as $\mathrm{s}$ rises from 5 to 10, so the best range for $\mathrm{s}$ is set to 5–10. In our experiments, $\mathrm{s}$ is set to 6.

When ${\mathsf{\lambda }}_1$ is 0.05, the indexes have a sharp change, as the spatial measurement begins to constrain clustering. Then, the indexes tend to be stable because of the limitation of the search scope. Indexes on Samson and Cuprite have little change, but it is different on Jasper Ridge. Specifically, AvgPI and SvdPI fall steadily and Scatt rises slowly. It is away from our requirement of regional homogeneity. Hence, ${\mathsf{\lambda }}_1$ is set to 0.1 to keep the stability as well as good performance of spatial clustering.

Figure 7 shows the results of spatial clustering with the parameters set above. For better viewing, part of each image is enlarged and shown in the red box.

$\mathrm{Pt}$ represents the percentage of pixels reserved from each partition for representative endmember generation. To analyze the impact of $\mathrm{Pt}$, an experiment is conducted. After completing the spatial clustering with $\mathrm{s}$ and ${\mathsf{\lambda }}_1$ being 6 and 0.1, respectively, choose pixels in each partition with $\mathrm{Pt}$ from 0.05 to 0.95 at an interval of 0.01, and the indexes are calculated each time. The results are shown in Figure 8.

Compared to Samson and Cuprite, indexes on Jasper Ridge have the most obvious change. It is caused by its poor spatial homogeneity. The increase in $\mathrm{Pt}$ results in the decline in $\mathrm{AvgPI}$ and $\mathrm{SvdPI}$ and the rise in $\mathrm{Scatt}$ as more mixed pixels within each partition are preserved. In terms of dispersion degree and spectral correlation, the smaller the $\mathrm{Pt}$, the better the updated partitions. Considering the prevalence of spectral variability, a small $\mathrm{Pt}$ cannot help dealing with it by average the reserved pixels, otherwise it brings errors. In our experiments, $\mathrm{Pt}$ is set to 0.4, and the good performance will be demonstrated in subsequent experimental results.

Moreover, compared to the indexes in Figure 6, $\mathrm{AvgPI}$ and $\mathrm{SvdPI}$ increase to above 0.99 and 0.85, respectively, and $\mathrm{Scatt}$ decreases to below 0.1. It further shows the important role of spectral purity analysis on regional homogeneity.

3.2.2. Spectral Clustering Analysis

There are two parameters involved in spectral clustering: the weighting factor ${\mathsf{\lambda }}_2$ and the number of clusters $\mathrm{k}$. The experimental basis of spectral clustering analysis is the completion of spatial clustering with s of 6 and ${\mathsf{\lambda }}_1$ of 0.1 and representative endmember generation with $\mathrm{Pt}$ of 0.4. To choose the best value for ${\mathsf{\lambda }}_2$ and $\mathrm{k}$, the following experiments are carried out. First, set ${\mathsf{\lambda }}_2$ to 0.4, calculate $\mathrm{Scatt}$, $\mathrm{AvgPI}$ and $\mathrm{SvdPI}$ on the spectral clustering results with $\mathrm{k}$ from $\mathrm{p}$ to 10*$\mathrm{p}$ at an interval of $\mathrm{p}$. The results are shown in Figure 9. Then, set $\mathrm{k}$ to 5*$\mathrm{p}$, and indexes are calculated on the k-means clustering results as ${\mathsf{\lambda }}_2$ increases from 0 to 1 with an interval of 0.05. The results are demonstrated in Figure 10.

It can be seen from the results that the response range of each index to $\mathrm{k}$ is greater than that to ${\mathsf{\lambda }}_2$, which indicates that $\mathrm{k}$ has a greater impact on spectral clustering than ${\mathsf{\lambda }}_2$. Similar to spatial clustering, indexes on Cuprite are the most stable compared to that on Samson and Jasper Ridge. This has relations with the different spectral contrast of endmembers in the datasets.

When $\mathrm{k}$ is below 5*$\mathrm{p}$, $\mathrm{AvgPI}$ and $\mathrm{SvdPI}$ rise rapidly, and then tend to stabilize. $\mathrm{Scatt}$ shows a fall trend, and roughly takes $\mathrm{k}$ of 5*$\mathrm{p}$ as the cut-off point. That is, when the number of clusters $\mathrm{k}$ is more than five times the number of endmembers, the spectral clustering keeps good performance with low dispersion degree and high spectral correlation within classes. At the same time, a large $\mathrm{k}$ may lead to over-clustering, as the same feature with spectral variability will be divided into different classes. Taking all this into consideration, set $\mathrm{k}$ to 5*$\mathrm{p}$. This number improves the computational efficiency of searching for the maximum simplex volume compared with other simplex-based algorithms.

With the increase in ${\mathsf{\lambda }}_2$, all indicators show a falling trend. That is, the reinforcement of spectral distance constraint enhances the central tendency but weakens the spectral shape similarity. To keep low dispersion degree and good spectral correlation, the best value of ${\mathsf{\lambda }}_2$ is set to 0.4.

3.3. Evaluation Metrics

Metrics are used to evaluate the performance of all test algorithms in terms of endmember extraction, abundance estimation and noise robustness.

$\mathrm{SAD}$ estimates the shape similarity between the spectra of the real endmember $\mathbf{z}$ and the extracted endmember ${\mathbf{z}}^{*}$ by calculating the angle of them. The smaller the value of $\mathrm{SAD}$, the more similar the two spectra.

$${\mathrm{SAD}=\cos }^{-1}\left(\frac{{\mathbf{z}}^{\mathbf{T}}{\mathbf{z}}^{*}}{\Vert \mathbf{z}\Vert ·\Vert {\mathbf{z}}^{*}\Vert }\right).$$

(14)

The root mean square error ($\mathrm{RMSE}$) calculates the error between the real abundance vector $\mathbf{m}$ and the estimated abundance vector ${\mathbf{m}}^{*}$ for each endmember. The fully constrained least squares (FCLS) algorithm is utilized to obtain the abundance map in this work [34]. A smaller $\mathrm{RMSE}$ means a higher unmixing accuracy with the extracted endmembers.

$$\mathrm{RMSE}=\sqrt{\frac{\Vert \mathbf{m} -{ \mathbf{m}}^{*}\Vert {}_2^2}{\mathrm{n}}}.$$

(15)

3.4. Results for Endmember Extraction

In this part, the performance of all algorithms on endmember extraction is compared. According to the previous parameter analysis, $\mathrm{s}$, ${\mathsf{\lambda }}_1$, $\mathrm{Pt}$, ${\mathsf{\lambda }}_2$ and $\mathrm{k}$ are set to 6, 0.1, 0.4, 0.4 and 5*$\mathrm{p}$ respectively. Figure 11, Figure 12 and Figure 13 give the comparison between the estimated and ground-truth endmember spectral signatures of Samson, Jasper Ridge and Cuprite, respectively. The $\mathrm{SAD}$ calculation results are shown in

Table 1. SAD of all test algorithms on Samson, Jasper Ridge and Cuprite datasets.

Samson
EEAs	VCA	SGA	AVMAX	MVSA	AMEE	ECSO	KMEE	FSPM	SENMAV	SPEW	CSVM
Soil	0.0404	0.0404	0.0404	0.0821	0.2853	0.0404	0.0404	0.0207	0.0404	0.0404	0.0109
Tree	0.0219	0.0407	0.0548	0.0489	0.0449	0.0713	0.4319	0.0495	0.0407	0.0407	0.0227
Water	0.1126	1.1483	0.1295	0.3712	0.2513	0.1552	0.1553	0.1299	0.1855	0.1305	0.0202
Average	0.0583	0.4098	0.0749	0.1674	0.1938	0.0890	0.2092	0.0667	0.0889	0.0705	0.0179
Jasper Ridge
EEAs	VCA	SGA	AVMAX	MVSA	AMEE	ECSO	KMEE	FSPM	SENMAV	SPEW	CSVM
Tree	0.1559	0.1559	0.1559	0.1837	0.1173	0.2030	0.0950	0.1979	0.1180	0.1686	0.0262
Water	0.2494	0.8953	0.2453	0.4035	0.1190	0.2540	0.7566	0.2399	0.2453	0.2430	0.0757
Soil	0.1336	0.1336	0.1336	0.2169	0.1302	0.1632	0.2299	0.2263	0.1353	0.1257	0.1231
Road	0.1069	0.1069	0.1069	0.3523	0.9162	0.2300	0.1792	0.1792	0.0443	0.0443	0.0148
Average	0.1614	0.3229	0.1604	0.2891	0.1957	0.2125	0.3152	0.2108	0.1357	0.1454	0.0599
Cuprite
EEAs	VCA	SGA	AVMAX	MVSA	AMEE	ECSO	KMEE	FSPM	SENMAV	SPEW	CSVM
Alunite	0.1023	0.0846	0.0846	0.1213	0.1062	0.0919	0.0919	0.0834	0.0919	0.1194	0.0931
Andradite	0.0880	0.0879	0.0706	0.1712	0.0817	0.0682	0.0658	0.0636	0.0717	0.0907	0.0698
Buddingtonite	0.0711	0.1008	0.0711	0.1537	0.1300	0.1093	0.1002	0.0824	0.1086	0.1072	0.1106
Dumortierite	0.0727	0.0761	0.0761	0.1420	0.1005	0.1195	0.1184	0.1019	0.0876	0.0684	0.0683
Kaolinite1	0.0904	0.0837	0.0837	0.2885	0.0758	0.1469	0.0837	0.0766	0.0852	0.0821	0.0853
Kaolinite2	0.0796	0.0806	0.0806	0.1995	0.0724	0.1235	0.0666	0.0740	0.0628	0.0590	0.0696
Muscovite	0.0816	0.0910	0.0878	0.3334	0.1945	0.0910	0.0816	0.1008	0.0910	0.1215	0.0878
Montmorillonite	0.0864	0.0899	0.0589	0.3252	0.1288	0.1493	0.0704	0.0796	0.0593	0.0921	0.0647
Nontronite	0.0700	0.0727	0.0703	0.4995	0.1289	0.1654	0.1248	0.0693	0.0939	0.0957	0.0804
Pyrope	0.0717	0.0677	0.1413	0.4173	0.0778	0.0874	0.0503	0.0622	0.0840	0.0641	0.0607
Sphene	0.2481	0.2374	0.2538	0.4798	0.0983	0.0581	0.0660	0.1174	0.2353	0.2840	0.1364
Chalcedony	0.1266	0.1621	0.1621	0.7734	0.2552	0.1602	0.2376	0.2027	0.1395	0.1301	0.1205
Average	0.0990	0.1029	0.1034	0.3254	0.1208	0.1142	0.0964	0.0928	0.1009	0.1095	0.0873

The best value in each line is bold.

.

To clearly show the comparison, the results of several test algorithms that perform better are drawn in the figures above, including VCA, AVMAX, SENMAV, SPEW and ECSO. From Figure 11, most algorithms can accurately extract soil and tree, but have a relatively large extraction error on water. CSVM achieves the best extraction accuracy on soil and water, as the curves almost coincide with the ground truth. Figure 12 shows the comparison for Jasper Ridge. According to the curves, all endmembers obtained by CSVM show the best consistency with ground truth in terms of shape and reflectance value. Figure 13 depicts the results for Cuprite. It is difficult to directly evaluate the algorithms through the curves since they all have difference with the ground truth in both shape and reflectance value. Therefore, quantitative analysis is necessary for further evaluation.

Table 1 shows the results of SAD. In Samson, CSVM obtains the best $\mathrm{SAD}$ for Soil and Water. SGA fails to extract water, as it gets a $\mathrm{SAD}$ more than 1, and KMEE fails to extract tree. CSVM provides best $\mathrm{SAD}$s on all endmembers in Jasper Ridge. SGA and KMEE are unable to extract water, and AMEE fails to extract road. All algorithms extract endmembers in Cuprite. CSVM gets the best results on dumortierite and chalcedony. CSVM does not achieve best extraction accuracy on all endmembers. For those that do not get the best $\mathrm{SAD}$, such as tree in Samson, muscovite in Cuprite and so on, CSVM obtains the results that are very close to the best values. Overall, CSVM provides the best averages for all datasets, corresponding to 0.0179, 0.0599 and 0.0873, respectively.

Compared to Cuprite, CSVM performs better on Samson and Jasper Ridge and greatly improves the extraction accuracy. The different performance on different datasets has relation to the spectral contrast of endmembers in the datasets. Cuprite has spectrally similar endmembers coupled with the effects of spectral variability, bringing difficulties to accurate extraction [35].

In conclusion, CSVM exhibits the best performance on endmember extraction, as it effectively alleviates the spectral variability. In addition, CSVM performs better for datasets with high spectral contrast of endmembers.

3.5. Results for Abundance Estimation

To evaluate the performance of all algorithms on abundance estimation, the endmembers obtained by these algorithms are adopted to generate abundance maps through the FCLS algorithm. RMSE is calculated between the estimated abundance vector and ground-truth abundance vector. The results are shown in

Table 2. RMSE of all test algorithms on Samson and Jasper Ridge datasets.

Samson
EEAs	VCA	SGA	AVMAX	MVSA	AMEE	ECSO	KMEE	FSPM	SENMAV	SPEW	CSVM
Soil	0.2696	0.4196	0.2636	0.3626	0.4205	0.2560	0.3441	0.2566	0.2619	0.2664	0.1794
Tree	0.2509	0.1671	0.2466	0.4537	0.4074	0.1869	0.3832	0.1821	0.2609	0.2515	0.2182
Water	0.4275	0.4520	0.4152	0.3251	0.2198	0.3516	0.3407	0.3489	0.4323	0.4237	0.3384
Average	0.3160	0.3462	0.3085	0.3804	0.3492	0.2648	0.3560	0.2625	0.3184	0.3139	0.2453
Jasper Ridge
EEAs	VCA	SGA	AVMAX	MVSA	AMEE	ECSO	KMEE	FSPM	SENMAV	SPEW	CSVM
Tree	0.1603	0.1592	0.1599	0.5046	0.2164	0.1232	0.3004	0.1170	0.2694	0.1426	0.1031
Water	0.2172	0.3224	0.2085	0.4221	0.0589	0.1541	0.2371	0.1836	0.2288	0.1356	0.1113
Soil	0.1402	0.1618	0.1300	0.3829	0.3290	0.1319	0.2262	0.2106	0.3207	0.1060	0.1107
Road	0.1112	0.1904	0.1223	0.2277	0.2268	0.1287	0.1738	0.2824	0.2277	0.0762	0.0728
Average	0.1572	0.2085	0.1552	0.3843	0.2078	0.1345	0.2344	0.1984	0.2616	0.1151	0.0995

The best value in each line is bold.

. Because there are no ground-truth abundance maps for Cuprite, only Samson and Jasper Ridge are used in this section. It is noted that the unmixing result is an indirect verification of EEAs, as it also relies on the abundance estimation algorithm.

From the results, CSVM obtains the best RMSE on soil and the best average for Samson. For Jasper Ridge, CSVM provides the best RMSE on tree, water and road and the best average. It indicates that the endmembers extracted by CSVM reduce the error of abundance estimation.

Therefore, compared with other EEAs, our method achieves better performance in hyperspectral unmixing considering endmember extraction and abundance estimation, comprehensively.

Figure 14 and Figure 15 demonstrate the comparison between estimated abundance maps using endmembers extracted by CSVM and the ground truth. As is shown in absolute error maps, CSVM obtains smaller estimation error on endmembers in Jasper Ridge than that in Samson, which means CSVM performs better on Jasper Ridge than on Samson. In addition, the results on both datasets show that the absolute error is large in areas with complex ground objects distribution, especially in the transition areas with different land cover types on both sides.

3.6. Results on Noisy Datasets

To verify the noise robustness of the test algorithms, set the SNR of the datasets to be 15 dB, 20 dB, 25 dB, 30 dB, 35 dB and 40 dB, then all algorithms are used to extract endmembers on the datasets. The average SAD between estimated endmembers and ground-truth endmembers are recorded in

Table 3. The average SAD of all test algorithms on noisy Samson, Jasper Ridge and Cuprite datasets.

Samson
SNR	15 dB	20 dB	25 dB	30 dB	35 dB	40 dB
VCA	0.3737	0.2469	0.1587	0.1083	0.0815	0.0756
SGA	0.4467	0.4264	0.4154	0.4112	0.3865	0.4100
AVMAX	0.3951	0.2467	0.1582	0.1105	0.0862	0.0743
MVSA	0.1707	0.1676	0.1685	0.1676	0.1674	0.1674
AMEE	1.0667	0.8954	0.5394	0.4595	0.1589	0.1493
ECSO	0.3486	0.2396	0.1505	0.1155	0.0792	0.0727
KMEE	0.3586	0.2491	0.2204	0.1952	0.1861	0.1836
FSPM	0.2499	0.2305	0.2284	0.0806	0.0701	0.0678
SENMAV	0.3896	0.2561	0.1511	0.1240	0.0873	0.0703
SPEW	0.3623	0.2478	0.1688	0.1022	0.0896	0.0748
CSVM	0.0556	0.0352	0.0337	0.0332	0.0306	0.0265
Jasper Ridge
SNR	15 dB	20 dB	25 dB	30 dB	35 dB	40 dB
VCA	0.3589	0.2831	0.2408	0.1897	0.1718	0.1561
SGA	0.3996	0.3189	0.3281	0.3113	0.3139	0.3233
AVMAX	0.3735	0.2466	0.1906	0.1694	0.1654	0.1626
MVSA	0.2977	0.2874	0.2778	0.2783	0.2836	0.2822
AMEE	1.2560	1.0092	0.5855	0.4614	0.3082	0.2004
ECSO	0.3475	0.2806	0.2734	0.2061	0.1884	0.2087
KMEE	0.3286	0.2541	0.2272	0.1677	0.1414	0.1378
FSPM	0.3003	0.2759	0.2491	0.2342	0.2288	0.2152
SENMAV	0.6655	0.2275	0.1702	0.1463	0.1277	0.1404
SPEW	0.6581	0.2821	0.2204	0.1606	0.1552	0.1488
CSVM	0.0762	0.0660	0.0673	0.0649	0.0609	0.0677
Cuprite
SNR	15 dB	20 dB	25 dB	30 dB	35 dB	40 dB
VCA	0.1920	0.1451	0.1193	0.1149	0.1016	0.1038
SGA	0.1924	0.1236	0.1196	0.1125	0.1166	0.1066
AVMAX	0.2084	0.1424	0.1186	0.1186	0.0996	0.0970
MVSA	0.3569	0.3750	0.4301	0.4020	0.3882	0.3670
AMEE	0.3406	0.2309	0.1662	0.1304	0.1198	0.1213
ECSO	0.2000	0.1810	0.1300	0.1202	0.1245	0.1014
KMEE	0.1918	0.1409	0.1222	0.1093	0.1065	0.1027
FSPM	0.1176	0.1130	0.1096	0.1065	0.1022	0.0953
SENMAV	0.2209	0.1473	0.1230	0.1079	0.1057	0.0993
SPEW	0.2191	0.1537	0.1091	0.1035	0.1011	0.0982
CSVM	0.1455	0.1121	0.1032	0.1033	0.0942	0.0955

The best value in each column is bold.

.

It can be seen from the results that with the increase in SNR, the average $\mathrm{SAD}$s commonly decline and achieve the best results at 40 dB. It shows that noise will distort the spectra of endmembers and reduce the extraction accuracy. There are special cases in all test algorithms. Since SGA cannot extract all endmembers as previous analyzed, the results of it have a slight change with the increasing of SNR. KMEE has anti-noise ability, but its average SADs vary greatly due to the unstable endmember extraction results. FSPM exhibits strong noise robustness, especially on the Cuprite dataset. As MVSA replaces positive hard constraint on abundance with a soft constraint of hinge loss type, it also shows good robustness to noise.

In addition, when SNR is below 30 dB, the results change obviously; when SNR is above 30 dB, the results have little change. It can be inferred that when SNR is set to 30 dB and above, the datasets may reach their best quality, thus limiting the extraction accuracy. Compared to Samson and Jasper Ridge, most algorithms seem to show better anti-noise performance on Cuprite, as the average SADs on Cuprite rise slowly with the decrease in SNR. This could be resulted from its low spectral contrast of endmembers.

Overall, AMEE exhibits weak noise robustness. It has larger averages than other algorithms when SNR is at a low level. CSVM obtains best extraction accuracy for all endmembers of datasets and achieves the best anti-noise performance.

3.7. Comparison on Processing Time

All the algorithms are implemented on Matlab R2019a with an Intel Core i5-10210U CPU at 1.60 GHz. The processing time of each test algorithm is shown in

Table 4. CPU processing time of all test algorithms on Samson, Jasper Ridge and Cuprite datasets (in seconds).

	Samson	Jasper Ridge	Cuprite
VCA	0.0386	0.0655	0.2258
SGA	0.4009	0.7169	17.1127
AVMAX	0.0641	0.0897	0.5941
MVSA	0.3790	0.6237	8.6719
AMEE	0.4283	0.1243	0.5960
ECSO	4.6723	12.4110	73.6082
KMEE	0.0497	0.1481	1.0280
FSPM	0.0975	0.1256	2.3582
SENMAV	0.7669	1.1362	28.0885
SPEW	0.1374	0.2175	1.9181
CSVM	0.7855	1.0408	7.6302

.

It is noted that ECSO is the most time-consuming algorithm, which requires a lot of computations. VCA is the most efficient method, as it takes a short time and provides high accuracy on endmember extraction. CSVM consumes a certain time due to the spatial clustering operation. This issue can be solved by the graphics processing units (GPU) parallel platform, as it shows superior performance in floating-point operations and programming. The superpixel segmentation provides possibility for parallel acceleration of CSVM.

4. Conclusions

In this paper, an endmember extraction method based on quadratic clustering is proposed to better handle spectral variability. CVSM first adopts SLIC-based spatial clustering to segment HSI into homogeneous partitions, and the average of pure pixels obtained by spectral purity analysis in each partition is taken as representative endmember, which reduces the effect of local-scope spectral variability. Then CVSM performs spectral clustering to merge spectrally related representative endmembers to further alleviate large-scope spectral variability, and the final endmembers are determined by simplex with maximum volume. CSVM is implemented from spatial and spectral dimension, which takes full of spatial and spectral information and effectively eliminates the effect of spectral variability.

Experiments are conducted to evaluate the performance of CSVM in terms of endmember extraction, abundance estimation, noise robustness and processing time. Through parameter analysis, the optimal parameters are first set for our experiments and future use. From the experimental results, the following can be concluded: (1) CSVM greatly improves the extraction accuracy by reducing the effect of spectral variability and performs better on datasets with high spectral contrast between endmembers; (2) CSVM provides better performance on abundance estimation with FCLS abundance estimation algorithm; (3) because of the average operations after clustering, CSVM also shows strong noise robustness; and (4) clustering makes CSVM consume lots of time, and it is one of our future research topics to further accelerate our method.