Label Noise Cleansing with Sparse Graph for Hyperspectral Image Classification

Abstract

In a real hyperspectral image classification task, label noise inevitably exists in training samples. To deal with label noise, current methods assume that noise obeys the Gaussian distribution, which is not the real case in practice, because in most cases, we are more likely to misclassify training samples at the boundaries between different classes. In this paper, we propose a spectral–spatial sparse graph-based adaptive label propagation (SALP) algorithm to address a more practical case, where the label information is contaminated by random noise and boundary noise. Specifically, the SALP mainly includes two steps: First, a spectral–spatial sparse graph is constructed to depict the contextual correlations between pixels within the same superpixel homogeneous region, which are generated by superpixel image segmentation, and then a transfer matrix is produced to describe the transition probability between pixels. Second, after randomly splitting training pixels into “clean” and “polluted,” we iteratively propagate the label information from “clean” to “polluted” based on the transfer matrix, and the relabeling strategy for each pixel is adaptively adjusted along with its spatial position in the corresponding homogeneous region. Experimental results on two standard hyperspectral image datasets show that the proposed SALP over four major classifiers can significantly decrease the influence of noisy labels, and our method achieves better performance compared with the baselines.

1. Introduction

Hyperspectral image (HSI), which includes numerous contiguous bands of spectral information of the observation, have been easily and cheaply acquired by various remote sensing sensors in recent years. HSI classification [1,2], which aims at the high-precision and pixel-level categorization of land use/land over types, has been significantly promoted for many practical applications [3,4], such as ecological monitoring, precision agriculture, and mineral exploration. As a result of the enduring and increasing demand of classification accuracy, the supervised paradigm of HSI classification benefiting from powerful knowledge of manual labels, is always one of the main topics of interest in the remote sensing community [5,6]. In the past decades, a number of effective HSI classification methods have been proposed based on multifarious supervised models, e.g., Bayesian model [7], neural networks [8], support vector machine (SVM) [9], sparse representation learning [10], collaborative representation learning [11,12], extreme learning machine (ELM) [13,14], and generative adversarial network [15,16]. They can generally be well-trained and provide satisfying results.

Commonly speaking, the availability of supervised models highly depends on the validity of manual labels. However, some unpredictable mislabeled samples (namely, label noise) naturally exist in real-world hyperspectral datasets when human labelers are involved, and the noise-polluted pixels would evidently decrease the performance of supervised classifiers [17]. To deal with the unavoidable label noise problem, researchers have proposed many useful methods [18,19,20,21,22,23,24] that mainly fall into three categories [25]: label noise-robust classification, label noise-tolerant classification, and label noise cleansing. Label noise-robust classification focuses on studying discriminative classifiers that are insensitive to label noise, label noise-tolerant classification is concerned with modeling the label noise simultaneously learning with the classifiers. Both approaches can be regarded as a style called “learning with the label noise,” which is always designed for specific classification models with sophisticated learning algorithms and thus has less generalization and scalability. By contrast, label noise cleansing is a preprocessing step, and it works on learning a universal noise filter to improve the label quality of training data for any classifiers. Therefore, this study focuses on the label noise cleansing style that is more general for HSI classification with noisy labels.

HSI classification in the presence of label noise is an exigent and challenging issue. However, few label cleansing methods are found when even considering the literature of generic image classification [26,27,28]. For example, Pelletier et al. [17] evaluated the influence of land cover label noise on classification performances. They demonstrated that the accuracies of two classic models, namely, SVM and random forest (RF), were sensitive to mislabeled training data. Unfortunately, they did not provide a solution to handle the label noise. Jiang et al. [29] proposed a random label propagation algorithm (RLPA), which can be the only relevant effort. They first constructed a probability transfer matrix to depict the affinity of hyperspectral pixels and then relabeled the training pixels by exploiting the label information from neighbors via random label propagation. The performance of several typical classifiers trained on relabeled samples had significant improvements over that of classifiers trained on initial samples with label noise. This approach assumed that noise obeys the Gaussian distribution, and noisy labels were generated by employing only random noise. However, the random noise cannot simulate the actual situation, because in most cases, the labelers probably misclassify hyperspectral pixels at the boundaries between different classes. Two kinds of label noise are more likely to take place during the practical labeling procedure [25]: random label noise and boundary label noise. On the one hand, hyperspectral pixels are so subtle that labelers may mislabel the samples unintentionally, which can increase the probability and randomness of noisy labels. On the other hand, diverse land use/land over types are often spatially staggered, and the labels of boundary pixels between adjacent surface types are harder to determine than those of other pixels. The neighborhood affinity is probably not robust when many noisy labels gather around the borders. Thus, a universal solution is needed to cope with the two types of label noise.

In this study, we propose a spectral–spatial sparse graph-based adaptive label propagation (SALP) algorithm to clean random label noise and boundary label noise for HSI classification. The framework shown in Figure 1 mainly includes two steps: First, we construct a spectral–spatial sparse graph to depict the affinity between pixels. Both spectral information and spatial constraint are important for calculating the similarity of different pixels [30,31,32,33], and sparse graph that explicitly considers the influence of data noise is insensitive to the gathered noise [34,35,36,37,38,39]. Therefore, a spectral–spatial sparse graph is adopted to describe the contextual correlations between pixels within same superpixel homogeneous region that are obtained by entropy rate superpixel segmentation (ESR) [40], and then a transfer matrix is produced to describe the transition probability between pixels. Second, we employ an adaptive label propagation algorithm to relabel the training samples. The spectral–spatial superpixel homogeneous region contains some useful prior knowledge that pixels with similar land use/land over types are often adjacent, and the central pixels in a superpixel homogeneous region are usually farther from the true border between different surface types more than other pixels (refer to the mapping results in Figure 1). We can adaptively relabel the pixels based on their spatial position in the corresponding homogeneous region. Hence, after randomly splitting training pixels into “clean” and “polluted,” an adaptive label propagation process is utilized to iteratively propagate the label information from “clean” to “polluted” based on the transfer matrix. The centered “polluted” pixels are relabeled by exploiting the label information from neighborhood “clean” samples, whereas the labels of other “polluted” pixels are adjusted by sparse “clean” pixels that locate in the same superpixel homogeneous region. Finally, the label propagation is carried out several times, and the final labels for each pixels are calculated based on a majority vote algorithm (MVA) [41]. The proposed SALP is tested on two real HSI databases, namely, Indian Pines and Salinas Scene, and our performance over four typical classifiers can significantly decrease the influence of noisy labels and exceed the baselines.

The main contributions of this study can be summarized as the following three aspects:

(1)

We carefully analyzed and examined the core issue and the influence of label noise for HSI classification, which is an useful guideline to present a label noise-polluted classification work.

(2)

A novel label noise cleansing method, namely, SALP algorithm, is proposed to deal with random label noise and boundary label noise.

(3)

Experimental results on two publicly practical datasets show that the proposed SALP over four major classifiers can obviously reduce the impact of noisy labels, and its performance can surpass the baselines in terms of overall accuracy (OA), average accuracy (AA), Kappa coefficient, and visual classification map.

This paper is organized as follows: Section 2 formulates and discusses the problem statement; Section 3 presents the methodology of the proposed SALP; Section 4 draws the experimental results and discussions; and Section 5 concludes this article.

2. Problem Statement

This section first presents the mathematical definitions of HSI classification in the presence of label noise, and then quantitatively discusses the impact of random label noise and boundary label noise.

Given a HSI composed of hundreds of contiguous spectral bands, some pixels with labels are regarded as training data, and the other pixels without labels are treated as testing data. When the HSI classification meets label noise, there are a certain level of noisy labels, including random label noise and boundary label noise, exist in the training pixels. To deal with the noisy labels, label noise cleansing is to relabel the initial labels to ensure that the performance of supervised classifiers using relabeled training data is better than that of using initial data.

Specifically, a set of training pixels in a D dimensional spectral feature space are denoted as $\mathbf{X}=\left\{{\mathbf{x}}_1,{\mathbf{x}}_2,\cdots ,{\mathbf{x}}_N\right\},{\mathbf{x}}_i\in {\mathbb{R}}^D$, i is an integer in the interval $[1,N]$. $\gamma =\left\{1,2,\cdots ,C\right\}$ depicts a label set, the class labels of $\left\{{\mathbf{x}}_1,{\mathbf{x}}_2,\cdots ,{\mathbf{x}}_N\right\}$ are denoted as $\left\{y_1,y_2,\cdots ,y_N\right\},y_j\in \gamma$, j is an integer in the interval $[1,C]$, and a matrix $\mathbf{Y}\in {\mathbb{R}}^{D\times C}$ is employed to denote the ground truth labels, where ${\mathbf{Y}}_{ij}=1$ if j is the label of ${\mathbf{x}}_i$. To model the label noise process, another matrix $\tilde{\mathbf{Y}}\in {\mathbb{R}}^{D\times C}$ is adopted to represent the labels with the noise of training pixels, and our goal is to forecast the label of ${\mathbf{x}}_l\in \mathbf{X}$ with the matrix $\tilde{\mathbf{Y}}$ in certain noise rate [42] $\rho$. And ${\rho }_{jk}$ that is the probability of one sample being marked by incorrect labels can be indicated as:

$$\begin{array}{c}\hfill {\rho }_{jk}=P\left({\tilde{\mathbf{Y}}}_{ik}=1|{\mathbf{Y}}_{ij}=1\right),j,k\in \left\{1,2,\cdots ,C\right\},\\ \hfill \forall j\ne k,i\in \left\{1,2,\cdots ,N\right\}.\end{array}$$

(1)

The influence of random label noise on HSI classification was examined in Jiang’s study [29], and results showed that random label noise can degenerate the performance of supervised classifiers. To further analyze the integrated impact of random label noise and boundary label noise, we test the noisy label-based algorithm (NLA), in which training samples and their corresponding noisy labels are directly used to train the classifiers. Four typical classifiers are utilized, namely, neighbor nearest (NN), SVM, RF, and ELM. Two standard hyperspectral datasets, i.e., the Indian Pines and Salinas, are adopted, and ground truth samples are randomly splited into training set and testing set. The label noise uses “both” setting that is a fusion of random label noise and boundary label noise. The noise rate is set as $\rho =\left\{0,0.1,0.2,\cdots ,0.9\right\}$ with an interval of 0.1. The OA is employed to measure the classification performance. More details about the sample setting, the label noise setting, and other experimental setup can be found in Section 4.2.

As shown in Figure 2, three phenomena occur: First, all classifiers have good performance when learning the classifiers without label noise, whereas their performance declines quickly along with the increase of $\rho$. Second, some classifiers may have good results at low noise rate. For example, ELM that pertains to single-hidden layer feed-forward neural network, can randomly generates the input weights and compute the output weights with a least squares solution. It is more computationally efficient meanwhile usually obtain similar or better generalization performance than traditional neural networks and SVM [13,43]. However, all classifiers are still sensitive to label noise especially when $\rho$ is high. Third, the performance of all classifiers is poor at a high noise rate and converges to a similarly low value. Above all, the random label noise and boundary label noise can seriously and negatively affect the availability of supervised classifiers. Thus, studying a general and effective label noise cleansing method for HSI classification is urgent.

3. Proposed Method

3.1. Overview of the Proposed Method

Label noise cleansing is performed to relabel the initial labeled training samples rather than the unlabeled samples adopted for unsupervised models [44,45]. Thus, the use of original labels is the key to the cleansing methodology. Although the process of pixel-level labeling is difficult and complicated, most labels are still credible because they are always generated by experts. Therefore, a intuitional idea is to relabel a pixel based on the majority of label information from its correlational pixels (e.g., neighbors [29]). The neighborhood affinity may be useful for cleansing the random label noise that is randomly distributed. However, different surface types are often so spatially staggered that the labels of boundary pixels are easily mislabeled, and the neighborhood affinity is probably less discriminative to eliminate boundary label noise.

To solve the problem, this study proposes SALP algorithm, and its core idea can be described as follows: First, the sparse graph that explicitly considers the influence of data noise is insensitive to the gathered label noise, and its sparsity is datum-adaptive instead of manually determining the size of neighborhood [46,47,48]. Second, spatial information is important for measuring the similarity of different pixels. A superpixel homogeneous region produced by superpixel image segmentation may provide a useful spatial constraint for graph construction [49,50]. Third, the centered pixels in a superpixel homogeneous region are often farther from the true border between different surface types more than other pixels. A good strategy is to adaptively adjust the label propagation for each pixel along with its spatial position in the corresponding homogeneous region. Therefore, we utilize a spectral–spatial sparse graph to depict the affinity between pixels within the same superpixel homogeneous region, and then iteratively relabel the training samples by exploiting the graph through an adaptive label propagation algorithm.

3.2. Spectral-Spatial Sparse Graph Construction

The core issue of graph construction is weight measurement between nodes, and a straightforward way to achieve this is to compute the spectral similarity between pixels. However, constructing a full graph for all pixels is complicated, and the commonly used similarities based on Euclidean distance and spectral angle mapper, etc., are not robust enough to handle the low inter-class and high intra-class spectral difference [51,52]. We notice that spatial information is helpful for differentiating the spectral features [43]. Thus, the initial HSI is segmented into many non-overlapping superpixel homogeneous regions, and then a sparse graph is generated based on the sparse spectral similarity with spatial constraint. The graphical illustration of spectral–spatial sparse graph is shown in Figure 3.

Firstly, following the superpixel segmentation procedure [29], $I_f$ is the first principal component of an HSI that contains hundreds of contiguous spectral bands, and it can be acquired through principal component analysis [53] so as to reduce the computational complexity. Then, a set of superpixel homogeneous regions $S=\left\{s_1,s_2,\cdots ,s_T\right\}$ is produced via ESR,

$$\begin{array}{c}\hfill I_f={\displaystyle \underset{k}{\overset{T}{\cup }}}{s}_k,s.t.s_k\cap s_g=\varnothing ,\\ \hfill \forall k,g\in \{1,2,\cdots ,T\},k\ne g,\end{array}$$

(2)

Secondly, a sparse graph is adopted to depict the contextual correlations between hyperspectral pixels. Instead of constructing a graph in the whole image, we employs a spatial constraint, in which the sparse graph is generated for the pixels in the same superpixel homogeneous region. Specifically, the training data $\mathbf{X}=\left\{{\mathbf{x}}_1,{\mathbf{x}}_2,\cdots ,{\mathbf{x}}_N\right\},{\mathbf{x}}_i\in {\mathbb{R}}^D$ are coded sparsely by ${\ell }_0$-norm optimization:

$$\underset{{\mathit{\alpha }}_i}{argmin}{∥{\mathit{\alpha }}_i∥}_0,s.t.{\mathbf{x}}_i={\mathbf{B}}_i{\mathit{\alpha }}_i,$$

(3)

where ${∥•∥}_0$ is the ${\ell }_0$-norm of a vector that counts the number of nonzero elements. ${\mathbf{B}}_i=\left\{{\mathbf{x}}_1,{\mathbf{x}}_2,\cdots ,{\mathbf{x}}_N,\mathbf{I}\right\}\in {\mathbb{R}}^{D\times (D+N-1)}$, ${\mathit{\alpha }}_i\in {\mathbb{R}}^{D+N-1}$. ALthough sparsity optimization is a nonconvex NP-hard problem, if the solution of ${\mathit{\alpha }}_i$ is sparse enough [54], Equation (3) can be relaxed by optimizing the following convex function:

$$\underset{{\mathit{\alpha }}_i}{argmin}{∥{\mathbf{x}}_i-{\mathbf{B}}_i{\mathit{\alpha }}_i∥}_2^2+\lambda {∥{\mathit{\alpha }}_i∥}_1,$$

(4)

where ${∥•∥}_1$ is the ${\ell }_1$-norm, which sums the absolute value of the vector, and can be solved as a linear program. $\lambda$ is used to weight the importance of minimizing ${\mathit{\alpha }}_i$.

Set sparse graph $G=\left\{\mathbf{X},\mathbf{W}\right\}$ with the $\mathbf{X}$ as the vertices set and the $\mathbf{W}$ as the weight matrix, and the edge weights ${\mathbf{W}}_{ij}$ from ${\mathbf{x}}_i$ to ${\mathbf{x}}_j$ can be denoted by:

$$\begin{array}{c}\hfill {\mathbf{W}}_{ij}=\left\{\begin{array}{c}{{\mathit{\alpha }}_i}^j,{\mathbf{x}}_i,{\mathbf{x}}_j\in s_k,\hfill \\ 0,{\mathbf{x}}_i\in s_k,{\mathbf{x}}_j\in s_g,\hfill \end{array}\right.\end{array}$$

(5)

where ${{\mathit{\alpha }}_i}^j$ is the j-th coefficient of ${\mathit{\alpha }}_i$. Generally, a large ${\mathbf{W}}_{ij}$ corresponds to the greater effect of ${\mathbf{x}}_i$ on ${\mathbf{x}}_j$ in the label propagation procedure. Notably, correlations may also exist between ${\mathbf{x}}_i$ and other pixels, and the overall affinity between associated pixels should be examined to calculate the transfer matrix ${\mathrm{T}}_{ij}$, which indicates the probability of label information traveling from ${\mathbf{x}}_i$ to ${\mathbf{x}}_j$ as follows:

$${\mathrm{T}}_{ij}=P\left(i\to j\right)=\frac{{\mathbf{W}}_{ij}}{{\sum }_{k=1}^N{\mathbf{W}}_{ik}}.$$

(6)

3.3. Adaptive Label Propagation

Label propagation is performed to deliver the label information from the correctly labeled pixels to the incorrect ones [55]. Although the ground truth is unknown, we can randomly divide the training data several times and then fuse all the results of label propagation based on the MVA. In each time, some training pixels that are randomly set as “clean” would preserve the labels, and the others are treated as “polluted” pixels that are unlabeled. To propagate the labels of “clean” samples, a common strategy that the “polluted” pixels absorb the label information from their neighbors [29] based on the transfer matrix $\mathrm{T}$, is sometimes useful to cope with the random label noise. However, boundary label noise is intensively located around the borders, and neighborhood affinity may bring about wrong labels for boundary pixels. Therefore, we update the label propagation adaptively for each pixel along with its spatial position in the corresponding homogeneous region.

Compared with the centered pixels in a superpixel homogeneous region, the pixels close to the homogeneous region border are more likely to be near the true boundary between different surface types. Thus, the labels of centered “polluted” pixels are altered based on the information from their neighborhood “clean” samples, whereas other “polluted” pixels are relabeled by exploiting the label information from sparse “clean” pixels that locate in the same superpixel homogeneous region. Specifically, a set of training pixels $\mathbf{X}$ is stochastically split into two parts: the “clean” subset ${\mathbf{X}}_L=\left\{{\mathbf{x}}_1,{\mathbf{x}}_2,\cdots ,{\mathbf{x}}_l\right\}$ with its label matrix ${\tilde{\mathbf{Y}}}_L=\tilde{\mathbf{Y}}\left(:,1:l\right)\in {\mathbb{R}}^{l\times C}$, l is the number of labeled pixels that is decided by the noise rate, $l=round\left(N\times \rho \right)$; the “polluted” subset ${\mathbf{X}}_U=\left\{{\mathbf{x}}_{l+1},{\mathbf{x}}_2,\cdots ,{\mathbf{x}}_N\right\}$ without labels. Then, the objective of adaptive label propagation is to iteratively forecast the label matrix ${\tilde{\mathbf{Y}}}_U$ of ${\mathbf{X}}_U$ by exploiting the transfer matrix $\mathrm{T}$.

Assuming that the label prediction matrix of all the training pixels is $\mathbf{F}=\left\{{\mathbf{f}}_1,{\mathbf{f}}_2,\cdots ,{\mathbf{f}}_N\right\}\in {\mathbb{R}}^{N\times C}$, the label of ${\mathbf{x}}_i$ at time $t+1$ is

$${\mathbf{f}}_i^{t+1}=\left\{\begin{array}{c}\theta {\displaystyle \sum _{{\mathbf{x}}_i\in s_k}}{\displaystyle \sum _{{\mathbf{x}}_j\in s_k^{i,neigh}}}{\mathrm{T}}_{ji}{\mathbf{f}}_j^t+\left(1-\theta \right){\tilde{\mathbf{Y}}}_{LU}^i,\mathrm{if}∥{\mathbf{x}}_i-s_k^{mean}∥<{\sigma }_k,\hfill \\ \theta {\displaystyle \sum _{{\mathbf{x}}_i,{\mathbf{x}}_j\in s_k}}{\mathrm{T}}_{ji}{\mathbf{f}}_j^t+\left(1-\theta \right){\tilde{\mathbf{Y}}}_{LU}^i,others,\hfill \end{array}\right.$$

(7)

where ${\tilde{\mathbf{Y}}}_{LU}^i$ is the i-th column of ${\tilde{\mathbf{Y}}}_{LU}=\left[{\tilde{\mathbf{Y}}}_L;{\tilde{\mathbf{Y}}}_U\right]$. $\theta$ is to balance the influence between the present label and other label information acquired from the referential pixels. We fix $\theta$ to 0.9 in all the experiments. ${\mathbf{f}}_j^t$ is the references label ${\mathbf{f}}_j$ at time t, the references $s_k^{i,neigh}$ are the neighbors of ${\mathbf{x}}_i$ when ${\mathbf{x}}_i$ is close to the mean of the superpixel homogeneous region, $s_k^{mean}$. Otherwise, ${\mathbf{f}}_j^t$ can be the labels of all other pixels within $s_k$. Parameter ${\sigma }_k$ is the variance of $s_k$. Following the optimization [56], Equation (7) can converge to:

$$\begin{array}{c}{\mathbf{F}}^{*}=\underset{t\to \infty }{lim}{\mathbf{F}}^t\hfill \\ =lim\theta \mathbf{T}{\mathbf{F}}^{t-1}+\left(1-\theta \right){\tilde{\mathbf{Y}}}_{LU}\hfill \\ =\left(1-\theta \right){\left(\mathbf{I}-\mathbf{T}\right)}^{-1}{\tilde{\mathbf{Y}}}_{LU},\hfill \end{array}$$

(8)

and the cleaned label of $x_i$ can be denoted as:

$$y_i^{*}=\underset{j}{argmax{\mathbf{F}}_{ji}^{*}}.$$

(9)

To reduce the error of random selection, the above process would be repeated for many times, and the final propagated label can be calculated by MVA. The pseudocodes of the proposed SALP is overall described as Algorithm 1.

Algorithm 1 The proposed SALP algorithm.

Input: A hyperspectral image $I_f$; The training pixels $\mathbf{X}=\left\{{\mathbf{x}}_1,{\mathbf{x}}_2,\cdots ,{\mathbf{x}}_N\right\}$ with their labels $\left\{y_1,y_2,\cdots ,y_N\right\}$; Parameters $\rho$ and $\theta$. Output: The cleaned label $\left\{y_1^{*},y_2^{*},\cdots ,y_N^{*}\right\}$. Generate the superpixel homogeneous region set S of $I_f$ by Equation (2); Construct the sparse graph by Equation (5); Calculate the transfer matrix $\mathit{T}$ by Equation (6); Update the label of ${\mathit{x}}_i$ at time $t+1$ based on $\mathit{T}$ by Equation (7); Acquire the cleaned label of ${\mathit{x}}_i$ by Equation (9);

4. Results and Discussions

4.1. Datasets

Two publicly standard HSI datasets are adopted to evaluate our method. The Indian Pines was captured by the AVIRIS sensor in northwestern Indiana, the scene of which includes two-thirds agriculture, and one-third forest or other natural perennial vegetation. The dataset consists of 145 × 145 pixels with 20 m per pixel and 220 bands in the wavelength range of 0.4–2.45 m. This study utilizes 200 bands for classification after removing 24 water absorption bands [15,29,57], and 10,249 labeled pixels with 16 different land cover types from the ground truth map. The gray image and the false-color composite image of the corresponding ground truth map are demonstrated in Figure 4.

The Salinas was collected by the 224-band AVIRIS sensor over Salinas Valley, California. Similar to the Indian Pines dataset, 20 water absorption bands were removed in this scene. This study employs the remaining 204 bands that are over 0.4–2.5 $\mathsf{\mu }$m of 512 × 217 pixels with a spatial resolution of 3.7 m, and 54,129 labeled pixels with 16 classes sampled from the ground truth map. The gray image and the false-color composite image of the corresponding ground truth map are demonstrated in Figure 5.

4.2. Experimental Setup

This section introduces the setup of our experiments, which mainly contains data setting, label noise generation, classification baselines, and evaluation metrics.

For the Indian Pines and Salinas datasets, pixels from the ground truth map are randomly divided into the training and testing samples. Specifically, on the Indian Pines dataset, 10 percent of pixels are randomly selected as training samples, and the others are regarded as testing samples. On the Salinas dataset, 50 training pixels from each class are randomly chosen for training, and the rest is for testing.

To comprehensively evaluate our method, we generate two kinds of label noise setting on training data through the procedure below. The first setting is random label noise (abbr. “random”). Although we focus on the more practical issue that some noisy labels may gather around the borders between different surface classes, and some of them distribute arbitrarily, existing methods always employ only random label noise. Therefore, hyperspectral pixels from the training data are randomly mislabeled at certain rate in the “random” setting. The second setting is the fusion of random label noise and boundary label noise (abbr. “both”). The boundary pixels are manually sought out in the ground truth label matrix, their k-nearest neighbors (KNN) are mislabeled to some extent, then the random noisy labels are acquired from the other pixels. The number of boundary noisy labels is same as that of random noisy labels in the “both” setting. The noise rate is set as $\rho =\left\{0.1,0.2,0.3,0.4,0.5\right\}$. We do not show the comparison results with $\rho$ > 0.5, because the labels of HSIs are provided by experts, and most of labels are always correct although a large amount of label noise probably exists in the practical applications.

Label noise cleansing is a process of HSI classification. We use four typical classifiers, namely, NN, SVM, RF, and ELM, to verify our method. In other words, we can evaluate the effectiveness of SALP based on the result of comparing the performance of classifier learning with the initial training data and the relabeled one. Three commonly-used metrics, i.e., the OA, AA, and the Kappa coefficient are used to measure the performance [57,58].

4.3. Results Comparison and Analysis with the “Random” Setting

Existing methods usually adopt only the random label noise. Thus, this section tests our method with the “random” setting in which label noise distributes randomly in the whole HSI. Three baselines of label noise cleansing are compared, which include NLA, isolation Forest (iForest) [59], and RLPA [29]. NLA is a basic baseline in which pixels with the corresponding noisy labels are directly used to train the classifiers. iForest is an anomaly detection algorithm that can be used to detect the noisy labels through the steps below: Many isolation trees are produced based on the sub-samples of training samples. Afterwards, the anomaly score for each sample is computed by analyzing the isolation trees, and those samples with scores that surpass the predefined threshold would be removed. RLPA is the state-of-the-art of label noise cleansing, and it is the first label cleansing solution to deal with the random noise label for HSI classification. The OA, AA, and Kappa coefficient of classification results over four typical classifiers on the Indian Pines and the Salinas datasets are shown in

Table 1. Comparisons of baselines and SALP over four typical classifiers with the “random” Setting on the Indian Pines dataset. The best results are bolded.

$\mathit{\rho }$	Classifier	OA [%]				AA [%]				Kappa Coefficient
		NLA	iForest	RLPA	SALP	NLA	iForest	RLPA	SALP	NLA	iForest	RLPA	SALP
0.1	NN	73.24	74.34	79.30	$\mathbf{80.49}$	69.55	69.50	72.63	$\mathbf{72.90}$	0.6966	0.7085	0.7635	$\mathbf{0.777}$
	SVM	84.21	83.73	$\mathbf{88.60}$	86.75	61.37	64.28	74.56	$\mathbf{75.44}$	0.8176	0.8122	$\mathbf{0.8695}$	0.8480
	RF	$\mathbf{80.29}$	79.11	79.72	78.69	$\mathbf{68.37}$	66.68	66.26	65.85	$\mathbf{0.7734}$	0.7597	0.7665	0.7538
	ELM	$\mathbf{91.35}$	90.11	91.66	90.07	$\mathbf{84.92}$	82.53	83.31	81.88	0.9012	0.8869	$\mathbf{0.9047}$	0.8844
0.2	NN	65.24	73.08	78.88	$\mathbf{80.46}$	62.73	66.14	72.32	$\mathbf{73.17}$	0.6086	0.6925	0.7587	$\mathbf{0.7766}$
	SVM	77.16	77.96	$\mathbf{88.01}$	86.85	54.93	62.78	74.67	$\mathbf{75.63}$	0.7340	0.7444	$\mathbf{0.8627}$	0.8491
	RF	78.41	74.03	$\mathbf{79.46}$	78.96	$\mathbf{67.37}$	61.26	66.12	66.64	0.7518	0.7006	$\mathbf{0.7635}$	0.7571
	ELM	88.65	83.30	$\mathbf{91.31}$	90.03	82.26	73.44	$\mathbf{83.23}$	82.50	0.8704	0.8082	$\mathbf{0.9006}$	0.8860
0.3	NN	57.46	70.95	78.02	$\mathbf{79.55}$	54.28	61.75	70.31	$\mathbf{71.74}$	0.5247	0.6688	0.7491	$\mathbf{0.7667}$
	SVM	71.10	76.82	$\mathbf{87.03}$	86.68	46.58	58.29	71.18	$\mathbf{74.23}$	0.6603	0.7315	0.8514	$\mathbf{0.8595}$
	RF	75.95	72.54	79.13	$\mathbf{79.27}$	64.40	58.52	65.55	$\mathbf{66.39}$	0.7243	0.6837	0.7598	$\mathbf{0.7609}$
	ELM	86.41	81.51	$\mathbf{90.59}$	89.70	77.28	68.72	80.94	$\mathbf{81.67}$	0.8447	0.7878	$\mathbf{0.8925}$	0.8822
0.4	NN	49.76	68.71	76.73	$\mathbf{79.15}$	47.82	60.12	69.66	$\mathbf{70.49}$	0.4421	0.6426	0.7348	$\mathbf{0.7622}$
	SVM	65.04	73.01	85.86	$\mathbf{86.52}$	39.00	57.25	71.21	$\mathbf{72.62}$	0.5851	0.6867	0.8381	$\mathbf{0.8533}$
	RF	72.43	69.59	$\mathbf{78.79}$	78.64	61.06	56.91	65.61	$\mathbf{64.78}$	0.6844	0.6494	$\mathbf{0.7562}$	0.7538
	ELM	82.93	76.91	89.62	$\mathbf{90.27}$	72.53	64.61	80.63	$\mathbf{81.46}$	0.8050	0.7345	0.8815	$\mathbf{0.8880}$
0.5	NN	40.56	64.64	72.91	$\mathbf{75.18}$	40.06	55.39	65.31	$\mathbf{65.69}$	0.3458	0.5967	0.6923	$\mathbf{0.7205}$
	SVM	61.91	68.90	82.16	$\mathbf{84.17}$	35.80	53.36	64.02	$\mathbf{65.86}$	0.5469	0.6388	0.7952	$\mathbf{0.8241}$
	RF	66.68	66.08	$\mathbf{76.75}$	76.39	57.04	53.39	63.47	$\mathbf{63.55}$	0.6212	0.6096	0.7329	$\mathbf{0.7292}$
	ELM	76.94	72.43	87.07	$\mathbf{87.11}$	67.74	58.94	76.69	$\mathbf{77.40}$	0.7373	0.6826	0.8522	$\mathbf{0.8540}$

and

Table 2. Comparisons of baselines and SALP over four typical classifiers with the “random” Setting on the Salinas dataset. The best results are bolded.

$\mathit{\rho }$	Classifier	OA [%]				AA [%]				Kappa Coefficient
		NLA	iForest	RLPA	SALP	NLA	iForest	RLPA	SALP	NLA	iForest	RLPA	SALP
0.1	NN	78.07	85.10	86.45	$\mathbf{87.08}$	83.95	91.72	93.01	$\mathbf{93.51}$	0.7579	0.8350	0.8497	$\mathbf{0.8566}$
	SVM	84.44	88.22	91.43	$\mathbf{92.81}$	91.74	93.14	95.45	$\mathbf{95.93}$	0.8272	0.8694	0.9047	$\mathbf{0.9200}$
	RF	86.97	86.71	88.09	$\mathbf{90.10}$	92.18	92.33	93.27	$\mathbf{94.52}$	0.8553	0.8526	0.8677	$\mathbf{0.8901}$
	ELM	92.69	90.54	92.96	$\mathbf{93.85}$	96.31	95.20	96.58	$\mathbf{96.84}$	0.9186	0.8949	0.9216	$\mathbf{0.9315}$
0.2	NN	70.22	84.93	85.89	$\mathbf{86.59}$	75.12	91.33	92.76	$\mathbf{93.34}$	0.6721	0.8331	0.8436	$\mathbf{0.8504}$
	SVM	85.87	88.24	91.13	$\mathbf{91.70}$	91.30	93.16	95.20	$\mathbf{95.70}$	0.8415	0.8694	0.9013	$\mathbf{0.9079}$
	RF	85.54	85.98	87.82	$\mathbf{89.15}$	90.54	91.66	93.12	$\mathbf{94.22}$	0.8395	0.8445	0.8648	$\mathbf{0.8775}$
	ELM	92.27	89.63	92.82	$\mathbf{93.41}$	95.92	94.59	96.49	$\mathbf{96.80}$	0.9139	0.8848	0.9201	$\mathbf{0.9267}$
0.3	NN	60.85	84.08	84.79	$\mathbf{86.5}$	65.44	90.26	92.14	$\mathbf{93.36}$	0.5710	0.8237	0.8318	$\mathbf{0.8512}$
	SVM	76.62	85.99	90.91	$\mathbf{91.63}$	89.47	91.48	95.12	$\mathbf{95.63}$	0.7437	0.8445	0.8989	$\mathbf{0.9069}$
	RF	82.59	84.69	87.12	$\mathbf{88.95}$	87.52	90.34	92.78	$\mathbf{94.21}$	0.8070	0.8301	0.8571	$\mathbf{0.8796}$
	ELM	91.34	88.22	92.56	$\mathbf{93.38}$	95.07	93.52	96.31	$\mathbf{96.65}$	0.9036	0.8692	0.9172	$\mathbf{0.9263}$
0.4	NN	53.99	83.72	83.27	$\mathbf{85.28}$	57.83	89.91	91.28	$\mathbf{92.65}$	0.4958	0.8197	0.8150	$\mathbf{0.8369}$
	SVM	77.52	84.79	90.08	$\mathbf{90.91}$	85.98	90.52	94.37	$\mathbf{95.34}$	0.7525	0.8313	0.8897	$\mathbf{0.9066}$
	RF	79.03	84.27	86.47	$\mathbf{88.88}$	83.62	90.01	92.54	$\mathbf{94.31}$	0.7675	0.8256	0.8500	$\mathbf{0.8767}$
	ELM	90.44	86.89	92.03	$\mathbf{93.17}$	94.17	92.69	96.10	$\mathbf{96.62}$	0.8936	0.8545	0.9114	$\mathbf{0.9241}$
0.5	NN	44.86	82.89	$\mathbf{80.79}$	80.32	47.17	88.53	89.22	$\mathbf{89.49}$	0.3978	$\mathbf{0.8104}$	0.7879	0.783
	SVM	75.03	84.52	89.28	$\mathbf{89.56}$	75.97	89.54	$\mathbf{94.45}$	93.85	0.7206	0.8281	0.8808	$\mathbf{0.8842}$
	RF	73.19	83.25	$\mathbf{85.20}$	85.10	77.06	88.62	91.50	$\mathbf{91.81}$	0.7034	0.8143	$\mathbf{0.8360}$	0.8352
	ELM	89.39	86.39	91.55	$\mathbf{91.99}$	93.26	91.70	$\mathbf{95.67}$	95.43	0.8819	0.8490	0.9061	$\mathbf{0.9109}$

.

On the basis of the results, three observations can be summarized as follows: (1) Compared with the NLA that directly learns with noisy data, SALP can significantly decrease the influence of random label noise. Our OA, AA, and Kappa coefficient outperform those of NLA at most noise rates, especially when high rates of label noise are present. For example, the improvements of OA at $\rho =0.5$ are 34.62%, 22.26%, and 9.71%, those of AA are 25.63%, 30.06%, 6.51%, and 9.66%, and those of Kappa coefficient are 37.47%, 16.36%, 13.18%, 2.9% for NN, SVM, RF, and ELM, respectively on the Indian Pines dataset, and greater improvements exist on the Salinas dataset. (2) Compared with the baselines of label noise cleansing, i.e., iForest and RLPA, SALP achieves better performance in most cases. The OA and Kappa coefficient of RLPA is slightly higher than that of SALP at low noise rates on the Indian Pines dataset, because the neighborhood constraint of RLPA still works well when the arbitrary noisy labels are almost dispersively distributed. However, our results exceed those of RLPA on Indian Pines at high noise rates, thereby probably inducing more agminated noisy labels even though a “random” setting exists. Moreover, the AA of SALP is evidently better than that of RLPA on both Indian Pines and Salinas dataset, and almost all of our results surpass baselines on the Salinas dataset. (3) Compared with ELM and RF that pertain to the label-robust learning paradigm and can be less influenced by noisy labels, NN and SVM are more sensitive to label noise. However, the performance degradations of NN and SVM based on SALP are usually less than those based on other baselines along with the increase in noise rate, it further demonstrates the effectiveness of our method for cleansing the label noise. The OA trend of SALP and RLPA (the best baseline) is shown in Figure 6. As can be seen, our method has a smoother curve and stable performance than RLPA on two datasets.

4.4. Results, Comparison and Analysis with the “both” Settings

In this section, we evaluate the effectiveness of SALP with the ”both” setting, in which training samples are polluted by random label noise and boundary label noise, which is a more practical case than the “random” setting. However, few studies employ boundary label noise. Thus, we carefully choose RLPA, which is the only label cleansing method for HSI classification and the NLA as baselines. The OA, AA, and Kappa coefficient of classification results over four typical classifiers on two hyperspectral datasets are shown in

Table 3. Comparisons of baselines and SALP over four typical classifiers with the “both” Setting on the Indian Pines dataset. The best results are bolded.

$\mathit{\rho }$	Classifier	OA [%]			AA [%]			Kappa Coefficient
		NLA	RLPA	SALP	NLA	RLPA	SALP	NLA	RLPA	SALP
0.1	NN	75.23	79.48	$\mathbf{79.99}$	67.80	72.14	$\mathbf{72.88}$	0.7071	0.7513	$\mathbf{0.7537}$
	SVM	83.93	85.12	$\mathbf{85.74}$	62.57	76.38	$\mathbf{77.42}$	0.8150	$\mathbf{0.8627}$	0.8362
	RF	78.24	78.90	$\mathbf{79.73}$	62.71	66.06	$\mathbf{66.99}$	0.7498	0.7568	$\mathbf{0.7664}$
	ELM	89.94	92.02	$\mathbf{92.43}$	81.42	82.63	$\mathbf{83.91}$	0.8850	0.9089	$\mathbf{0.9135}$
0.2	NN	65.86	78.45	$\mathbf{78.53}$	63.31	71.81	$\mathbf{71.91}$	0.6169	0.7566	$\mathbf{0.7714}$
	SVM	78.51	84.85	$\mathbf{87.82}$	55.69	73.41	$\mathbf{76.45}$	0.7539	0.8481	$\mathbf{0.8602}$
	RF	$\mathbf{78.72}$	77.65	76.73	62.06	$\mathbf{65.37}$	64.70	$\mathbf{0.7560}$	0.7422	0.7314
	ELM	89.06	$\mathbf{91.93}$	91.35	80.48	81.22	$\mathbf{82.46}$	0.8752	$\mathbf{0.9077}$	0.9009
0.3	NN	59.63	77.55	$\mathbf{77.93}$	56.04	71.27	$\mathbf{71.56}$	0.5368	0.7446	$\mathbf{0.7485}$
	SVM	71.41	84.03	$\mathbf{86.89}$	45.48	72.47	$\mathbf{73.45}$	0.6693	0.8395	$\mathbf{0.8495}$
	RF	75.20	77.98	$\mathbf{78.69}$	63.96	64.15	$\mathbf{64.92}$	0.7149	0.7475	$\mathbf{0.7557}$
	ELM	87.99	$\mathbf{91.60}$	90.45	79.40	$\mathbf{82.41}$	82.13	0.8625	$\mathbf{0.8939}$	0.8909
0.4	NN	49.58	77.46	$\mathbf{79.36}$	44.97	71.97	$\mathbf{72.10}$	0.4417	0.7439	$\mathbf{0.7553}$
	SVM	64.24	83.18	$\mathbf{85.45}$	36.40	64.37	$\mathbf{67.35}$	0.5783	0.8129	$\mathbf{0.8325}$
	RF	72.11	78.62	$\mathbf{78.45}$	58.67	$\mathbf{65.93}$	64.98	0.6823	$\mathbf{0.7544}$	0.7524
	ELM	82.57	$\mathbf{91.06}$	90.16	69.94	81.73	$\mathbf{82.04}$	0.8010	$\mathbf{0.8922}$	0.8875
0.5	NN	42.42	72.54	$\mathbf{74.50}$	42.18	66.20	$\mathbf{68.17}$	0.3665	0.6892	$\mathbf{0.7117}$
	SVM	54.47	80.80	$\mathbf{85.76}$	28.55	61.92	$\mathbf{64.45}$	0.4615	0.7898	$\mathbf{0.8313}$
	RF	65.53	78.05	$\mathbf{78.79}$	53.61	62.89	$\mathbf{63.03}$	0.6061	0.7468	$\mathbf{0.7490}$
	ELM	78.10	89.98	$\mathbf{90.74}$	67.51	81.26	$\mathbf{81.78}$	0.7505	0.8856	$\mathbf{0.8881}$

and

Table 4. Comparisons of baselines and SALP over four typical classifiers with the “both” Setting on the Salinas dataset. The best results are bolded.

$\mathit{\rho }$	Classifier	OA [%]			AA [%]			Kappa Coefficient
		NLA	RLPA	SALP	NLA	RLPA	SALP	NLA	RLPA	SALP
0.1	NN	78.20	$\mathbf{86.98}$	86.62	84.43	$\mathbf{93.63}$	93.52	0.7602	0.8528	$\mathbf{0.8556}$
	SVM	90.19	92.18	$\mathbf{92.26}$	94.09	95.85	$\mathbf{95.97}$	0.8906	0.9082	$\mathbf{0.9087}$
	RF	86.75	87.65	$\mathbf{87.65}$	92.23	93.25	$\mathbf{93.25}$	0.8640	0.8632	$\mathbf{0.8632}$
	ELM	92.96	93.86	$\mathbf{93.95}$	96.68	$\mathbf{96.99}$	96.97	0.9281	0.9309	$\mathbf{0.9310}$
0.2	NN	70.96	85.75	$\mathbf{86.15}$	76.77	93.01	$\mathbf{93.34}$	0.6803	0.8368	$\mathbf{0.8416}$
	SVM	89.49	91.78	$\mathbf{92.10}$	91.79	$\mathbf{95.87}$	95.78	0.8831	0.9014	$\mathbf{0.9121}$
	RF	84.44	86.48	$\mathbf{87.00}$	90.37	$\mathbf{92.93}$	92.86	0.8272	0.8503	$\mathbf{0.8514}$
	ELM	92.02	93.79	$\mathbf{93.79}$	95.67	96.12	$\mathbf{96.38}$	0.9113	0.9191	$\mathbf{0.9237}$
0.3	NN	62.78	84.27	$\mathbf{86.06}$	66.43	92.16	$\mathbf{92.88}$	0.5911	0.8287	$\mathbf{0.844}$
	SVM	83.4	91.18	$\mathbf{91.78}$	88.29	94.98	$\mathbf{95.32}$	0.8136	0.9021	$\mathbf{0.9065}$
	RF	81.60	86.11	$\mathbf{86.59}$	88.21	92.29	$\mathbf{93.14}$	0.7967	0.8544	$\mathbf{0.8605}$
	ELM	90.76	$\mathbf{93.83}$	93.44	93.96	96.10	$\mathbf{96.50}$	0.8970	0.9116	$\mathbf{0.9267}$
0.4	NN	53.23	83.97	$\mathbf{84.29}$	58.89	91.21	$\mathbf{92.77}$	0.4891	0.8080	$\mathbf{0.8365}$
	SVM	75.60	91.25	$\mathbf{91.60}$	83.71	94.31	$\mathbf{95.40}$	0.7301	$\mathbf{0.9027}$	0.9004
	RF	77.40	85.99	$\mathbf{86.26}$	82.90	91.95	$\mathbf{92.99}$	0.7501	0.8450	$\mathbf{0.8477}$
	ELM	90.70	$\mathbf{93.68}$	93.27	93.47	96.08	$\mathbf{96.26}$	0.8989	0.9144	$\mathbf{0.9247}$
0.5	NN	42.82	78.87	$\mathbf{80.74}$	46.64	87.47	$\mathbf{88.93}$	0.3788	0.7521	$\mathbf{0.7680}$
	SVM	72.84	87.00	$\mathbf{91.05}$	79.77	92.66	$\mathbf{94.03}$	0.6986	0.8563	$\mathbf{0.8704}$
	RF	75.00	$\mathbf{86.58}$	86.24	77.21	91.05	$\mathbf{92.16}$	0.7222	0.8313	$\mathbf{0.8569}$
	ELM	88.90	$\mathbf{93.59}$	93.16	93.16	$\mathbf{95.71}$	95.51	0.8767	0.9073	$\mathbf{0.9100}$

.

Four conclusions can be drawn based on the results: (1) The proposed SALP achieves evidently better results than NLA in terms of OA, AA, and Kappa coefficient, which means that our method can effectively clean the fusion of random label noise and boundary label noise. (2) The performance of the proposed SALP algorithm surpasses that of RLPA, which is the state-of-the-art of label noise cleansing for HSI classification in most situations, especially when high noise rates exist, and the improvements of results with the “both” setting are more obvious compared with those with the “random” setting, it states that our SALP is useful and suitable to deal with both random label noise and boundary label noise. (3) For RF and ELM that are insensitive to label noise, SALP can perserve their robustness. For more sensitive NN and SVM, our method can slow down the degradation of classification performance along with the increase in $\rho$. The OA trend of SALP and RLPA is compared in Figure 7, and the OA curves of our method are steadier those of RLPA accompanied by the rise in the noise rate. (4) To further present the visualization results, the classification maps on two noise levels ($\rho =0.1$ and $\rho =0.5$) are shown in Figure 8 and Figure 9 for the Indian Pines, and in Figure 10 for the Salinas dataset. The classification maps of SALP result in improved labels especially for the boundary pixels between different surface types and achieve a smoother effect for the hyperspectral pixels in some tiny surface classes, e.g., the northeast of Indian Pines.

4.5. Further Discussion

Compared with KNN graph. Figure 3 shows the graphical difference between full graph and spectral–spatial sparse graph. To present a quantitative comparison, we test a commonly-used full graph that is constructed based on KNN, and the results in the “both” setting are shown in Figure 11. We observe that the performace of SALP is significantly better than that of KNN graph at any noise rates. Moreover, the results of KNN graph regress rapidly as the noise rate grows, especially when the classifiers adopt NN and SVM. The robustness demonstrates the effectiveness of SALP, and KNN graph is sensitive to label noise for HSI classification.

Influence of boundary label noise on SALP. The proportion of random label noise and boundary label noise is equal in the previous “both” setting. In this section, we evaluate the OA trend of SALP when the proportion is altered. As shown in Figure 12, two phenomenon can be observed: First, the curves with different proportions of boundary label noise on two datasets are steady, which further proves the superiority of SALP. Second, there exists an increasing trend of OA until the proportion of boundary label noise is more than 0.8, especially on the Indina Pines dataset. On one hand, it demonstrates that SALP is usually suitable for coping with boundary label noise. On the other hand, Indian Pines has more small surface patches than Salinas, and noisy labels probably submerge these patches if there are too much boundary label noise, which may reduce the effectiveness of SALP.

5. Conclusions

Label noise is an unavoidable and urgent issue for HSI classification. Compared with the state-of-the-art that employs only the random label noise, this study proposes a SALP algorithm to handle a more practical noise condition in which the real HSI classification task is more likely to involve random label noise and boundary label noise. The SALP first constructs a robust sparse graph based on the spectral similarity between hyperspectral pixels with spatial constraint, and then adaptively propagates the label information from “clean” samples to “polluted” samples by exploiting the graph. Three major conclusions can be drawn based on extensive experimental results on two standard public datasets: First, our method always achieves better performance than baselines in terms of OA, AA and Kappa coefficient with either “random” setting or “both” setting, thereby indicating that SALP can effectively clean random label noise and boundary label noise. Second, SALP can slow down the degradation of classification performance along with the increase in label noise rate for those label noise-sensitive classifiers, e.g., NN and SVM. Third, the performance of our method is usually steady and sometimes improved when the proportion of boundary label noise is increased in the “both” setting, thus demonstrating that SALP can handle the gathered boundary label noise.