Next Article in Journal
Geospatial Google Street View with Virtual Reality: A Motivational Approach for Spatial Training Education
Previous Article in Journal
An Internet-Based GIS Platform Providing Data for Visualization and Spatial Analysis of Urbanization in Major Asian and African Cities
Article Menu
Issue 8 (August) cover image

Export Article

ISPRS Int. J. Geo-Inf. 2017, 6(8), 258; https://doi.org/10.3390/ijgi6080258

Article
Spatial-Spectral Graph Regularized Kernel Sparse Representation for Hyperspectral Image Classification
1
School of Internet of Things Engineering, Jiangnan University, Wuxi 214122, China
2
School of Computer Science, Nanjing University of Science and Technology, Nanjing 210094, China
*
Author to whom correspondence should be addressed.
Academic Editor: Wolfgang Kainz
Received: 12 June 2017 / Accepted: 18 August 2017 / Published: 22 August 2017

Abstract

:
This paper presents a spatial-spectral method for hyperspectral image classification in the regularization framework of kernel sparse representation. First, two spatial-spectral constraint terms are appended to the sparse recovery model of kernel sparse representation. The first one is a graph-based spatially-smooth constraint which is utilized to describe the contextual information of hyperspectral images. The second one is a spatial location constraint, which is exploited to incorporate the prior knowledge of the location information of training pixels. Then, an efficient alternating direction method of multipliers is developed to solve the corresponding minimization problem. At last, the recovered sparse coefficient vectors are used to determine the labels of test pixels. Experimental results carried out on three real hyperspectral images point out the effectiveness of the proposed method.
Keywords:
classification; sparse representation; hyperspectral image; kernel; regularization

1. Introduction

Hyperspectral imaging sensors capture digital images in hundreds of narrow and contiguous spectral bands spanning the visible to infrared spectrum. Each pixel of a hyperspectral image can be represented by a vector whose entries correspond to various spectral-band responses. Different materials usually reflect electromagnetic energy differently at specific wavelengths. The wealth of spectral information of hyperspectral images promotes the development of many application domains, such as military [1], agriculture [2], and mineralogy [3]. Among the information processing procedures of these applications, classification is an important one, where pixels are assigned to one of the available classes according to a set of given training pixels. Various classifiers have been developed for hyperspectral image classification and have shown good performance. Some popular classifiers include support vector machine (SVM) [4], multinomial logistic regression [5], and sparse representation classification (SRC) [6].
Hyperspectral images usually have large homogeneous regions within which the neighboring pixels consist of the same type of materials and have similar spectral characteristics. Previous works have highlighted that hyperspectral image classification should not only focus on analyzing spectral features, but also incorporate the information of spatially-adjacent pixels. For instance, the authors in [7] incorporate the spectral and spatial information into SVM by taking advantage of the composite kernel technique. In [5], the spatial information is incorporated into multinomial logistic regression by some spatial feature extraction techniques. In [6], a joint sparsity model is introduced to SRC, where the neighborhood pixels are assumed to have the same labels. Very recently, some deep models (e.g., stacked autoencoder [8] and convolutional neural network [9,10,11,12]) have been proposed for the spatial-spectral classification of hyperspectral images and have shown good performance in terms of accuracy and flexibility.
Among the aforementioned classifiers, SRC is an excellent one and can directly assign a class label to a test pixel without a training process. Although SRC can achieve good performance in hyperspectral image classification [6], it is hard to classify the data that is not linearly separable. To overcome this drawback, kernel SRC (KSRC) is proposed in [13] to capture the nonlinear similarity of features. In view of this, this paper considers using KSRC for the classification of hyperspectral images. However, KSRC is also a pixel-wise classifier, which does not consider the spectral similarity of spatially-adjacent pixels. In the literature, some methods have been proposed to incorporate the spatial information into KSRC. In [13], a joint sparsity model is introduced to KSRC, which can be seen as a kernel extension of [6]. In [14], the spatial-spectral information is incorporated into KSRC by taking advantage of the neighboring filtering kernel technique. In [15], a centralized quadratic constraint that describes the spectral similarity of spatially-adjacent pixels is appended to the sparse recovery model of KSRC.
This paper considers incorporating the spatial-spectral information of hyperspectral images in the regularization framework of KSRC. Different from the relevant methods, a spatial-spectral graph is designed to describe the contextual information of hyperspectral images, where the spectral similarity of spatially-adjacent pixels (i.e., nodes) is measured by the weight of the edge. In addition, the acquisition of labeled pixels in most hyperspectral scenes is often an expensive procedure. Thus, the number of training pixels that can be used for hyperspectral image classification is often very limited. To deal with this problem, we assume that the training pixels are labeled by experts and then exploit the wealth of these training pixels. In detail, a spatial location constraint is introduced to capture the prior knowledge of the location information of training pixels and is thereby appended to the sparse recovery model of KSRC. As for the final classification procedure, the sparse coefficient vectors obtained by solving the corresponding regularization problem are used to determine the labels of test pixels.
The remainder of this paper is organized as follows. Section 2 briefly reviews the formulations of KSRC. In Section 3, we present the formulations of the proposed SSGL method and the corresponding optimization algorithm. The effectiveness of the proposed method is demonstrated in Section 4 by conducting experiments on three real hyperspectral images. Finally, Section 5 discusses and concludes this paper.

2. KSRC

This section briefly introduces the notations of kernel sparse representation for hyperspectral image classification. As with the assumption of SRC, KSRC also assumes that the features belonging to the same class approximately lie in the same low-dimensional subspace [16,17]. Suppose the given hyperspectral scene includes C classes, and that there exists a feature mapping function ϕ which maps an L-band test pixel x R L and a J-sized training dictionary A = [ a 1 , , a J ] R L × J to the high-dimensional feature space: x ϕ ( x ) , A Φ ( A ) = [ ϕ ( a 1 ) , , ϕ ( a J ) ] , where the columns of A are composed of J training pixels from all classes. Then, for an unknown test pixel x R L , it can be represented as follows:
ϕ ( x ) = Φ ( A ) s ,
where s R J denotes an unknown sparse coefficient vector.
Suppose the dictionary A is given; the unknown sparse coefficient vector s can be obtained by solving the following optimization problem:
min s 1 2 | | ϕ ( x ) Φ ( A ) s | | 2 2 + λ | | s | | 1 ,
where λ > 0 is used to control the sparsity of s . Then, the class label of x can be determined by the following classification rule:
class ( x ) = arg min c = 1 , , C | | ϕ ( x ) Φ ( A ) δ c ( s ) | | 2 ,
where δ c ( · ) is the characteristic function that selects coefficients related with the c-th class and makes the rest zero.
Since all ϕ mappings used in KSRC occur in the form of inner products, the kernel function K can be defined for any two pixels x i R L and x j R L as follows:
K ( x i , x j ) = ϕ ( x i ) , ϕ ( x j ) .
Some commonly-used kernel functions are: linear kernel ( K ( x i , x j ) = x i , x j ), polynomial kernel ( K ( x i , x j ) = ( x i , x j + 1 ) d , d Z + ), and Gaussian radial basis function (RBF) kernel ( K ( x i , x j ) = exp ( γ | | x i x j | | 2 2 ) , γ R + ). In this paper, only the RBF kernel is considered. After introducing (4) into (2), the corresponding optimization problem can be rewritten as follows:
min s 1 2 s T Q s s T p + λ | | s | | 1 + Const ,
where Const = ( 1 / 2 ) K ( x , x ) is a constant that can be dropped in the optimization problem (5), Q = Φ ( A ) , Φ ( A ) is a J × J positive semi-definite matrix with entry Q i j = K ( a i , a j ) , and p = [ K ( a 1 , x ) , , K ( a J , x ) ] T R J . Analogously, the classification rule (3) can be rewritten as follows:
class ( x ) = arg min c = 1 , , C δ c T ( s ) Q δ c ( s ) 2 δ c T ( s ) p .

3. Proposed Approach

3.1. Proposed Model

Suppose the observed hyperspectral image X consists of I pixels { x i R L } i = 1 I ; i.e., X = [ x 1 , , x I ] R L × J . Then, the corresponding sparse recovery model of (2) for the observed hyperspectral image X can be written as follows:
min S 1 2 | | Φ ( X ) Φ ( A ) S | | F 2 + λ | | S | | 1 , 1 ,
where | | · | | F is the Frobenius norm, | | S | | 1 , 1 = j , i | S j i | , Φ ( X ) = [ ϕ ( x 1 ) , , ϕ ( x I ) ] denotes the corresponding data of X under the mapping function ϕ and S = [ s 1 , , s I ] R J × I is the unknown sparse coefficient matrix, with s i R J being the sparse coefficient vector of pixel x i . Similarly, the corresponding sparse recovery model of (5) can be written as follows:
min S 1 2 Tr ( S T Q S ) Tr ( S T P ) + λ | | S | | 1 , 1 ,
where the constant term is dropped and Tr ( · ) denotes the trace of a matrix, P = Φ ( A ) , Φ ( X ) = [ p 1 , , p I ] R J × I with entry P i j = K ( a i , x j ) . The corresponding classification rule of (6) can be written as follows:
class ( x i ) = arg min c = 1 , , C δ c T ( s i ) Q δ c ( s i ) 2 δ c T ( s i ) p i , i = 1 , 2 , , I .
The matrix X is not just a set of pixels; it is essentially a three-dimensional image; since for any i in the complete set I = { 1 , 2 , , I } , s i is the sparse coefficient vector of pixel x i . Then, the matrix S can also be seen as a three-dimensional image. That is to say, the spatial relationship between every two pixels x i and x j is also suitable for that of the sparse coefficient vectors s i and s j . It is natural to incorporate the spatial information of X by enforcing S . However, directly enforcing S is too strict; it does not consider the variation of training pixels within each class. In view of this, the summation of sparse coefficients that belong to each class is considered. Specifically, the spatial information is incorporated by enforcing TS , where T R C × I is defined as:
T c j = 1 if class ( a j ) = c 0 else , c = 1 , 2 , , C ; j = 1 , 2 , , J .
Let us define a spatial-spectral graph as G = { V , E , W } , where V = [ v 1 , , v I ] and E are the sets of vertices and edges, respectively, and W R I × I is a weight matrix on the graph. In this paper, every node v i is connected to its eight spatially-adjacent nodes (see Figure 1), and the weight between v i and v i is defined as follows:
W i j = exp ( β | | x ¯ i x ¯ j | | 2 ) + 10 6 if v i and v i are connected 0 else , i , j
where β > 0 and x ¯ i and x ¯ j are the pixels of the first three principal components of the hyperspectral image X . It is apparent that for any two similar nodes, a large weight will be given. In this paper, the graph-based spatially-smooth constraint term is defined as follows:
S ( TS ) = Tr ( TS ) L ( TS ) T ,
where L = D W is the graph Laplacian and D is a diagonal matrix whose entries are column sums of W (i.e., D i i = j W i j ). After appending (12) to the sparse recovery model (8), we can get the following optimization problem:
min S 1 2 Tr ( S T Q S ) Tr ( S T P ) + λ | | S | | 1 , 1 + α 2 Tr ( TS ) L ( TS ) T ,
where α > 0 . In order to effectively work with limited training pixels, the prior knowledge of the location information of training pixels is included by a spatial location constraint term. The final sparse recovery model can be written as follows:
min S 1 2 Tr ( S T Q S ) Tr ( S T P ) + λ | | S | | 1 , 1 + α 2 Tr ( TS ) L ( TS ) T + ι 0 ( TS Λ T ) ,
where Λ I is a set with S Λ being the corresponding coefficient matrix of the labeled pixels A and ι 0 ( · ) denotes the indicator function; i.e., ι 0 ( · ) is zero if the given matrix is equal to zero, and infinity otherwise. By using the location information of training pixels in the model (14), these training pixels can be treated as anchor samples whose coefficients are fixed throughout the classification process, and the graph-based spatially-smooth constraint can spread their label information to their neighbors until a global stable state is achieved on the whole data set.

3.2. Optimization Algorithm

By introducing two variables M R J × I and N R C × I , the proposed optimization model (14) can be rewritten as follows:
min S , M , N f ( S , M , N )      s . t . M = S , N = TS .
where
f ( S , M , N ) = 1 2 Tr ( S T Q S ) Tr ( S T P ) + λ | | M | | 1 , 1 + α 2 Tr ( N L N T ) + ι 0 ( N Λ T ) .
The optimization problem (15) accords with the framework of the alternating direction method of multipliers (ADMM) [18,19]. The augmented Lagrangian function of (15) can be written as follows:
L ( S , M , N , H , B ) = f ( S , M , N ) + μ 2 | | S M H | | F 2 + μ 2 | | TS N B | | F 2 ,
where μ > 0 is the penalty parameter and H R J × I and B R C × I are two auxiliary variables. The corresponding iteration procedure can be written as follows:
S t = arg min S L ( S , M t 1 , N t 1 , H t 1 , B t 1 ) M t = arg min M L ( S t , M , N t 1 , H t 1 , B t 1 ) N t = arg min N L ( S t , M t , N , H t 1 , B t 1 ) H t = H t 1 ( S t M t ) B t = B t 1 ( TS t N t )
where t > 0 is the iteration number. The first step of (18) is to solve the S -subproblem, which can be derived as:
S t = ( Q + μ I + μ T T T ) 1 ( P + μ ( M t 1 + H t 1 ) + μ T T ( N t 1 + B t 1 ) ) ,
where I is the identity matrix. The second step of (18) is to solve the M -subproblem, which is the well-known soft threshold [20]:
M t = soft ( S t H t 1 , λ / μ ) ,
where soft ( · , τ ) denotes the component-wise application of the soft-threshold function y = sign ( y ) max { | y | τ , 0 } . The third step of (18) is to solve a linear system, which can be written as:
min N 1 2 N ( TS t B t 1 ) F 2 + α 2 μ Tr ( N L N T ) + ι 0 ( N Λ T ) .
If removing the last term, the solution of this system can be written as:
N t = ( TS t B t 1 ) α μ L + I 1 .
Let Λ ¯ be the complementary set of Λ under the complete set I . Then, the optimization problem (21) can be rewritten as:
min N Λ ¯ 1 2 [ N Λ N Λ ¯ ] ( T [ S Λ t S Λ ¯ t ] [ B Λ t 1 B Λ ¯ t 1 ] ) F 2 + α 2 μ Tr [ N Λ N Λ ¯ ] L Λ Λ L Λ Λ ¯ L Λ ¯ Λ L Λ ¯ Λ ¯ [ N Λ N Λ ¯ ] T ,
where L Λ Λ ¯ = L Λ ¯ Λ T R J × ( I J ) and N Λ = T . The solution of (23) can be derived as:
N Λ ¯ = TS Λ ¯ t B Λ ¯ t 1 α μ N Λ L Λ Λ ¯ α μ L Λ ¯ Λ ¯ + I 1 .
For both (22) and (24), one has to solve a large linear system. Fortunately, L is a sparse and diagonally dominant matrix, and one can take the Gauss–Seidel method to get approximate solutions to these two systems. The last two steps of (18) are designed to update the auxiliary variables. The procedure of the proposed algorithm is detailed as follows:
  • Input: A training dictionary A R L × J and a hyperspectral data matrix X R L × I .
  • Choose β , and compute the weight matrix W according to (11).
  • Select the γ parameter for the RBF kernel and compute the matrices Q and P .
  • Set t = 1 ; choose μ , λ , α , S 1 , M 1 , N 1 , H 1 , B 1 .
  • Repeat
  • Compute S t , M t , N t , H t , B t using (18).
  • t = t + 1 .
  • Until some stopping criterion is satisfied.
  • Output: The estimated label of x i using (6), i = 1 , , I .

3.3. Analysis and Comparison

The proposed spatial-spectral graph regularization with location (SSGL) method is designed by appending two spatial-spectral constraint terms to the sparse recovery model of KSRC. One is a graph-based spatially-smooth constraint, and the other is a spatial location constraint. When using the two terms, the following two issues should be mentioned.
  • The graph-based spatially-smooth constraint is proposed by measuring the spatial relationship between every two spatially-adjacent pixels. If using this term, the test pixels should be arranged in the form of images.
  • The spatial location constraint is exploited by integrating the location information of anchor samples, which are assumed to be taken from the test area and labeled by experts.
As described in the introduction part, deep models attract a lot of attention and provide competitive classification results very recently. Although the proposed SSGL method and the deep learning methods belong to two distinct techniques, we show the difference between them considering a further improvement of this work. In Table 1, we compare the proposed SSGL method and the deep learning methods in the respects of parameter, training sample, computational cost, flexibility, and accuracy.

4. Experimental Results

4.1. Datasets

To test the performance of the proposed method, three real hyperspectral images have been considered.
(1) Airborne Visible/Infrared Imaging Spectrometer (AVIRIS) Indian Pines: The first image in our experiments deals with the standard AVIRIS image taken over northwest Indiana’s Indian Pine test site in June 1992. There are 220 bands in the image, covering the wavelength range of 0.4–2.5 μ m. The spectral and spatial resolutions are 10 nm and 17 m, respectively. This image consists of 145 × 145 pixels and 16 ground truth classes ranging from 20–2468 pixels in size. The false color image and the ground truth map are shown in Figure 2. We removed 20 noisy bands covering the region of water absorption and worked with 200 spectral bands. In the experiments, about 5% of the labeled pixels were randomly chosen for training, and the rest were used for testing, as shown in Table 2.
(2) Reflective Optics System Imaging Spectrometer (ROSIS) University of Pavia: This dataset is an urban image acquired by the ROSIS sensor, with spectral coverage ranging from 0.43–0.86 μ m. The ROSIS sensor has a spatial resolution of 1.3 m per pixel, with 115 spectral bands. This image, with a size of 610 × 340 pixels, contains 103 spectral bands after the removal of noisy bands. There are nine ground truth classes of interest. The false color image and the ground truth map are shown in Figure 3. In the experiments, we randomly chose 40 samples per class for training and used the rest for testing, as shown in Table 3.
(3) AVIRIS Kennedy Space Center: This image was collected by the AVIRIS sensor over the Kennedy Space Center, Florida, in March 1996. There are 224 bands in the image, covering the wavelength range of 0.4–2.5 μ m. The spectral and spatial resolutions are 10 nm and 18 m, respectively. This image acquired over an area of 512 × 614 pixels contains 176 spectral bands after removing water absorption and low signal-to-noise bands. Thirteen ground-truth classes ranging from 105–927 pixels in size that occur in this scene are defined for this image. The false color image and the ground truth map are shown in Figure 4. In the experiments, 5% of the labeled pixels were randomly chosen for training, and the rest were used for testing, as shown in Table 4.

4.2. Model Development and Experimental Setup

In the experiments, we compared the proposed SSGL method with four widely-used spatial-spectral classification methods based on the KSRC classifier. The first one is the majority voting (MV) method that performs a pixel-wise KSRC followed by majority voting within superpixel regions [21,22], where the superpixel regions are obtained by the entropy rate superpixel (ERS) algorithm [23]. The second one is the kernel joint SRC (KJSRC) method using kernel joint sparse representation classification [13,24], where the superpixel-based adaptive neighborhoods are used [24]. The third one is the condition random fields (CRF) method that combines pixel-wise KSRC and superpixel segmentation by using condition random fields [25]. The last one is the composite kernel with extended multi-attribute profile features (CKEMAP) method that integrates a composite kernel [7] into KSRC for the incorporation of spatial-spectral information, where the spatial features are extracted by EMAP [26,27]. In order to show the influence of the proposed two terms, we included a degeneration version (i.e., SSG) of SSGL for comparison, where the location information constraint term was dropped. Moreover, the pixel-wise KSRC [17] that only uses spectral features was also included as a baseline classifier. The corresponding optimization problems of the compared methods were all solved by ADMM. For the pixel-wise KSRC, the parameter γ for the RBF kernel was experimentally set to two for the AVIRIS Indian Pines dataset, 0.5 for the ROSIS University of Pavia dataset, and 0.125 for the AVIRIS Kennedy Space Center dataset, and the parameters λ and μ were empirically set to 10 4 and 10 3 , respectively. For further details, one can refer to the existing literature [14]. Considering that all compared methods are based on the KSRC classifier, their parameters brought by KSRC were experimentally set to be the same as those of the pixel-wise KSRC. For MV, the additional parameters brought by the ERS segmentation were carefully optimized by reference to [28]. For both KJSRC and CRF, the parameters brought by the ERS segmentation were fixed to be the same as those of MV, and the remaining parameters were obtained by cross-validation. For CKEMAP, the EMAP features were extracted from the first three principal components of the hyperspectral image, and the area and standard deviation attributes were considered to build EMAP, as reported in [29]. More specifically, for the area attribute the following values were selected for references: 49, 169, 361, 625, 961, 1369, 1849, and 2401. For the standard deviation, the EMAPs were built according to the following reference values with respect to the mean of the individual features: 2.5%, 5%, 7.5%, and 10%. As for the weight parameter and the kernel parameters in the composite kernel framework [7], they were obtained by cross-validation. For the proposed method, μ was set to 10 4 , α was set to 1, and β was experimentally set to 50 for the AVIRIS Indian Pines dataset and 100 for both the ROSIS University of Pavia dataset and the AVIRIS Kennedy Space Center dataset. As for SSG, the same parameters were used as SSGL.
Before the following experiments, the original data were scaled in the range [0, 1]. The classification accuracies are assessed with OA (overall accuracy), AA (average accuracy), and KA (kappa coefficient of agreement). The quantitative measures were obtained by averaging ten runs to avoid any bias induced by random sampling. All experiments were carried out in a 64-b quad-core CPU 2.40-GHz processor with 8 GB of memory.

4.3. Numerical and Visual Comparisons

Table 5 summarizes the class-specific and global classification accuracies for the AVIRIS Indian Pines dataset. The processing times in seconds are also included for reference. Figure 5 shows the corresponding classification maps. From Table 5, it can be seen that all of the spatial-spectral methods yielded higher classification accuracies when compared to the pixel-wise KSRC. SSGL gave the highest global and most of the best class-specific accuracies, and CKEMAP performed the second highest. For the proposed methods SSG and SSGL, the improvement of SSGL over SSG was significant. From Figure 5, it can be seen that the maps of the spatial-spectral methods contain more homogeneous regions when compared with the map of the pixel-wise KSRC. The misclassified regions of SSGL and CKEMAP were smaller than those of the others.
The class-specific and global classification accuracies for the ROSIS University of Pavia dataset are provided in Table 6, as well as the processing times. The corresponding classification maps are shown in Figure 6. From the table, it is clear that SSGL performed best followed by SSG, and the performances of the spatial-spectral methods were better than those of the pixel-wise method. The numerical comparisons are confirmed by inspecting the classification maps. Table 7 shows the class-specific and global classification accuracies and the processing times for the AVIRIS Kennedy Space Center dataset. It can be seen that Table 7 reveals almost the same results as Table 5. The classification maps shown in Figure 7 confirm the numerical comparisons further.
In this set of experiments, it can be seen that SSGL performed best in all scenes. This demonstrates the superiority of the spatial location constraint as well as the graph-based spatially-smooth constraint. However, for SSG, it is more suitable to classify the ROSIS scene. This is because the spatial resolution of the ROSIS scene is higher than those of the two AVIRIS scenes, and thus the spectral similarity of the spatially-adjacent pixels of which is more suitable for SSG to measure.

4.4. Analysis of Parameters

Among the parameters of the proposed method, β was set experimentally and differently for the three given datasets. In this set of experiments, we evaluated the influence of β by varying it from 10–200. Figure 8 shows the classification accuracies for both SSGL and SSG when applied to the three given datasets. It can be seen that when β is small, the classification performance of SSG increases monotonically as β increases; and the results typically saturate for large β s. When β is relatively large, the accuracies of SSGL decrease slightly. In all cases, we can choose β in a wide optimal region. Thus, the choice of β is robust.

4.5. Influence of Anchor Samples

In SSGL, the spatial location constraint is built on the prior knowledge of the location information of training samples. That is to say, all training samples are treated as anchor samples. In this set of experiments, we built the spatial location constraint term by using different percentages of training sample to show the influence of anchor samples. Figure 9 shows the classification accuracies for SSGL when applied to the three given datasets. From the figure, we can see that the accuracies increase monotonically as the percentage increases, but the accuracies reach the optimal values asymptotically when a moderate number of anchor samples are used. Thus, we can build the spatial location term by using a relatively small number of anchor samples.

4.6. Different Numbers of Training Samples

In this set of experiments, we tested the performance of the compared classification methods in an ill-posed scenario, where different numbers of training pixels are used. Specifically, for both the AVIRIS Indian Pines dataset and the AVIRIS Kennedy Space Center dataset, we randomly chose 1–20% of the labeled pixels per class for training and the remaining pixels for testing. For the classes with limited labeled pixels, at least two pixels were chosen for training. For the ROSIS University of Pavia dataset, we built training sets by randomly choosing 10, 20, 40, 60, 80, and 100 training pixels per class and used the rest as test sets. Figure 10 shows the mean value and standard deviation of OA as a function of the number of training pixels for the three given datasets. From the figures, it can be seen that the proposed SSGL outperformed the compared methods in almost all cases. As expected, the performance of the spatial-spectral methods was better than that of the pixel-wise KSRC, and the classification accuracies decreased when the number of training pixels was reduced. When compared with the AVIRIS scene, SSG is more suitable for the ROSIS scene.

5. Discussion and Conclusions

A regularized kernel sparse representation method for spatial-spectral classification of hyperspectral images has been presented in this paper, where the spatial-spectral information of hyperspectral images is incorporated by appending two spatial-spectral constraint terms to the sparse recovery model of KSRC. One is the graph-based spatially-smooth constraint, which is designed to measure the spectral similarity of spatially-adjacent pixels. The other is the spatial location constraint, which is used to exploit the wealth of training pixels and thereby to reduce the cost of acquiring a large number of labeled pixels. By introducing the two constraints, the spatial and label information of the hyperspectral pixels—especially that of the training pixels—can be spread to their neighbors, which is very useful for the discrimination of each class. Experiments conducted on three real hyperspectral images have demonstrated that the proposed method can yield accurate classification results. Although the results obtained by the proposed method are very encouraging in hyperspectral image classification, further enhancements such as more robust constraint terms and more reasonable regularization strategies should be pursued in future developments.

Acknowledgments

This work was supported by the National Natural Science Foundation of China (Grant number 61601201) and the Natural Science Foundation of Jiangsu Province (Grant numbers BK20160188 and BK20150160). The authors would like to thank Prof. D. Landgrebe from Purdue University for providing the AVIRIS Indian Pines dataset and Prof. P. Gamba from the University of Pavia, Italy, for providing the ROSIS University of Pavia dataset.

Author Contributions

Jianjun Liu wrote the paper. Jianjun Liu and Jinlong Yang analyzed the data. Jianjun Liu and Zhiyong Xiao conceived of and designed the experiments. Yufeng Chen performed the experiments. Yunfeng Chen contributed analysis tools.

Conflicts of Interest

The authors declare no conflict of interest. The founding sponsors had no role in the design of the study; in the collection, analyses or interpretation of the data; in the writing of the manuscript; nor in the decision to publish the results.

Abbreviations

The following abbreviations are used in this manuscript:
SVMsupport vector machines
SRCsparse representation classification
KSRCkernel sparse representation classification
SSGLspatial-spectral graph regularization with location information
RBFGaussian radial basis function
ADMMalternating direction method of multipliers
AVIRISAirborne Visible/Infrared Imaging Spectrometer
ROSISReflective Optics System Imaging Spectrometer
MVmajority voting
KJSRCkernel joint sparse representation classification
CRFcondition random fields
CKEMAPcomposite kernel with extended multi-attribute profile features
EMAPextended multi-attribute profile
ERSentropy rate superpixel
SSGspatial-spectral graph regularization
OAoverall accuracy
AAaverage accuracy
KAkappa coefficient of agreement

References

  1. Manolakis, D.; Shaw, G. Detection algorithms for hyperspectral imaging applications. IEEE Signal Process. Mag. 2005, 19, 29–43. [Google Scholar] [CrossRef]
  2. Datt, B.; Mcvicar, T.R.; Van Niel, T.G.; Jupp, D.L.B. Preprocessing EO-1 Hyperion hyperspectral data to support the application of agricultural indexes. IEEE Trans. Geosci. Remote Sens. 2003, 41, 1246–1259. [Google Scholar] [CrossRef]
  3. Horig, B.; Kuhn, F.; Oschutz, F.; Lehmann, F. HyMap hyperspectral remote sensing to detect hydrocarbons. Int. J. Remote Sens. 2001, 22, 1413–1422. [Google Scholar] [CrossRef]
  4. Melgani, F.; Bruzzone, L. Classification of hyperspectral remote sensing images with support vector machines. IEEE Trans. Geosci. Remote Sens. 2004, 42, 1778–1790. [Google Scholar] [CrossRef]
  5. Li, J.; Huang, X.; Gamba, P.; Bioucas-Dias, J.M.; Zhang, L.; Benediktsson, J.A.; Plaza, A. Multiple feature learning for hyperspectral image classification. IEEE Trans. Geosci. Remote Sens. 2015, 53, 1592–1606. [Google Scholar] [CrossRef]
  6. Chen, Y.; Nasrabadi, N.M.; Tran, T.D. Hyperspectral image classification using dictionary-based sparse representation. IEEE Trans. Geosci. Remote Sens. 2011, 49, 3973–3985. [Google Scholar] [CrossRef]
  7. Camps-Valls, G.; Gomez-Chova, L.; Munoz-Mari, J.; Vila-Frances, J.; Calpe-Maravilla, J. Composite kernels for hyperspectral image classification. IEEE Geosci. Remote Sens. Lett. 2006, 3, 93–97. [Google Scholar] [CrossRef]
  8. Chen, Y.; Lin, Z.; Zhao, X.; Wang, G.; Gu, Y. Deep Learning-Based Classification of Hyperspectral Data. IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens. 2014, 7, 2094–2107. [Google Scholar] [CrossRef]
  9. Makantasis, K.; Karantzalos, K.; Doulamis, A.; Doulamis, N. Deep supervised learning for hyperspectral data classification through convolutional neural networks. In Proceedings of the IEEE International Geoscience and Remote Sensing Symposium, Milan, Italy, 26–31 July 2015. [Google Scholar]
  10. Chen, Y.; Jiang, H.; Li, C.; Jia, X.; Ghamisi, P. Deep feature extraction and classification of hyperspectral images based on convolutional neural networks. IEEE Trans. Geosci. Remote Sens. 2016, 54, 6232–6251. [Google Scholar] [CrossRef]
  11. Wan, L.; Liu, N.; Huo, H.; Fang, T. Selective convolutional neural networks and cascade classifiers for remote sensing image classification. Remote Sens. Lett. 2017, 8, 917–926. [Google Scholar] [CrossRef]
  12. Wang, J.; Luo, C.; Huang, H.; Zhao, H.; Wang, S. Transferring Pre-Trained Deep CNNs for Remote Scene Classification with General Features Learned from Linear PCA Network. Remote Sens. 2017, 9, 225. [Google Scholar] [CrossRef]
  13. Chen, Y.; Nasrabadi, N.M.; Tran, T.D. Hyperspectral Image Classification via Kernel Sparse Representation. IEEE Trans. Geosci. Remote Sens. 2013, 51, 217–231. [Google Scholar] [CrossRef]
  14. Liu, J.; Wu, Z.; Wei, Z.; Xiao, L.; Sun, L. Spatial-spectral kernel sparse representation for hyperspectral image classification. IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens. 2013, 6, 2462–2471. [Google Scholar] [CrossRef]
  15. Yuan, H.; Tang, Y.Y.; Lu, Y.; Yang, L.; Luo, H. Hyperspectral Image Classification Based on Regularized Sparse Representation. IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens. 2014, 7, 2174–2182. [Google Scholar] [CrossRef]
  16. Wright, J.; Yang, A.Y.; Ganesh, A.; Sastry, S.S.; Ma, Y. Robust face recognition via sparse representation. IEEE Trans. Pattern Anal. Mach. Intell. 2008, 31, 210–227. [Google Scholar] [CrossRef] [PubMed]
  17. Gao, S.; Tsang, I.W.H.; Chia, L.T. Kernel Sparse Representation for Image Classification and Face Recognition. In Proceedings of the 11th European Conference on Computer Vision: Part IV, Heraklion, Greece, 5–11 September 2010; pp. 1–14. [Google Scholar]
  18. Goldstein, T.; Osher, S. The split Bregman method for L1-regularized problems. SIAM J. Imaging Sci. 2009, 2, 323–343. [Google Scholar] [CrossRef]
  19. Yang, J.; Zhang, Y. Alternating direction algorithms for l1-problems in compressive sensing. SIAM J. Sci. Comput. 2011, 33, 250–278. [Google Scholar] [CrossRef]
  20. Combettes, P.L.; Wajs, V.R. Signal Recovery by Proximal Forward-Backward Splitting. Multiscale Model. Simul. 2005, 4, 1168–1200. [Google Scholar] [CrossRef]
  21. Tarabalka, Y.; Chanussot, J.; Benediktsson, J.A. Segmentation and classification of hyperspectral images using watershed transformation. Pattern Recogition 2010, 43, 2367–2379. [Google Scholar] [CrossRef]
  22. Liu, J.; Shi, X.; Wu, Z.; Xiao, L.; Xiao, Z.; Yuan, Y. Hyperspectral image classification via region-based composite kernels. In Proceedings of the 2016 IEEE International Geoscience and Remote Sensing Symposium (IGARSS), Beijing, China, 10–15 July 2015. [Google Scholar]
  23. Liu, M.Y.; Tuzel, O.; Ramalingam, S.; Chellappa, R. Entropy rate superpixel segmentation. In Proceedings of the 2011 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Colorado Springs, CO, USA, 20–25 June 2011. [Google Scholar]
  24. Feng, J.; Cao, Z.; Pi, Y. Polarimetric Contextual Classification of PolSAR Images Using Sparse Representation and Superpixels. Remote Sens. 2014, 6, 7158–7181. [Google Scholar] [CrossRef]
  25. Roscher, R.; Waske, B. Superpixel-based classification of hyperspectral data using sparse representation and conditional random fields. In Proceedings of the IEEE International Geoscience and Remote Sensing Symposium, Quebec City, QC, Canada, 13–18 July 2014. [Google Scholar]
  26. Mura, M.D.; Benediktsson, J.A.; Waske, B.; Bruzzone, L. Extended profiles with morphological attribute filters for the analysis of hyperspectral data. Inte. J. Remote Sens. 2010, 31, 5975–5991. [Google Scholar] [CrossRef]
  27. Mura, M.D.; Benediktsson, J.A.; Waske, B.; Bruzzone, L. Morphological Attribute Profiles for the Analysis of Very High Resolution Images. IEEE Trans. Geosci. Remote Sens. 2010, 48, 3747–3762. [Google Scholar] [CrossRef]
  28. Fang, L.; Li, S.; Kang, X.; Benediktsson, J.A. Spectral–Spatial Classification of Hyperspectral Images with a Superpixel-Based Discriminative Sparse Model. IEEE Trans. Geosci. Remote Sens. 2015, 53, 4186–4201. [Google Scholar] [CrossRef]
  29. Li, J.; Marpu, P.R.; Plaza, A.; Bioucas-Dias, J.M.; Benediktsson, J.A. Generalized Composite Kernel Framework for Hyperspectral Image Classification. IEEE Trans. Geosci. Remote Sens. 2013, 51, 4816–4829. [Google Scholar] [CrossRef]
Figure 1. Illustration of the spatially-adjacent nodes. The yellow one denotes the given node and the black ones denote the eight spatially-adjacent nodes.
Figure 1. Illustration of the spatially-adjacent nodes. The yellow one denotes the given node and the black ones denote the eight spatially-adjacent nodes.
Ijgi 06 00258 g001
Figure 2. Airborne Visible/Infrared Imaging Spectrometer (AVIRIS) Indian Pines data set. (a) RGB composite image of three bands. (b) Ground-truth map.
Figure 2. Airborne Visible/Infrared Imaging Spectrometer (AVIRIS) Indian Pines data set. (a) RGB composite image of three bands. (b) Ground-truth map.
Ijgi 06 00258 g002
Figure 3. Reflective Optics System Imaging Spectrometer (ROSIS) University of Pavia dataset. (a) RGB composite image of three bands. (b) Ground-truth map.
Figure 3. Reflective Optics System Imaging Spectrometer (ROSIS) University of Pavia dataset. (a) RGB composite image of three bands. (b) Ground-truth map.
Ijgi 06 00258 g003
Figure 4. AVIRIS Kennedy Space Center dataset. (a) RGB composite image of three bands. (b) Ground-truth map.
Figure 4. AVIRIS Kennedy Space Center dataset. (a) RGB composite image of three bands. (b) Ground-truth map.
Ijgi 06 00258 g004
Figure 5. Classification maps and overall classification accuracies (in parentheses) obtained for the AVIRIS Indian Pines dataset using different classification methods. (a) KSRC (81.49), (b) MV (90.93), (c) KJSRC (92.79), (d) CRF (93.92), (e) CKEMAP (95.39), (f) SSG (88.30), (g) SSGL (96.44).
Figure 5. Classification maps and overall classification accuracies (in parentheses) obtained for the AVIRIS Indian Pines dataset using different classification methods. (a) KSRC (81.49), (b) MV (90.93), (c) KJSRC (92.79), (d) CRF (93.92), (e) CKEMAP (95.39), (f) SSG (88.30), (g) SSGL (96.44).
Ijgi 06 00258 g005
Figure 6. Classification maps and overall classification accuracies (in parentheses) obtained for the ROSIS University of Pavia dataset using different classification methods. (a) KSRC (83.44), (b) MV (93.71), (c) KJSRC (94.55), (d) CRF (95.56), (e) CKEMAP (96.67), (f) SSG (97.57), (g) SSGL (98.28).
Figure 6. Classification maps and overall classification accuracies (in parentheses) obtained for the ROSIS University of Pavia dataset using different classification methods. (a) KSRC (83.44), (b) MV (93.71), (c) KJSRC (94.55), (d) CRF (95.56), (e) CKEMAP (96.67), (f) SSG (97.57), (g) SSGL (98.28).
Ijgi 06 00258 g006
Figure 7. Classification maps and overall classification accuracies (in parentheses) obtained for the AVIRIS Kennedy Space Center dataset using different classification methods. (a) KSRC (89.86), (b) MV (93.30), (c) KJSRC (92.96), (d) CRF (93.49), (e) CKEMAP (97.43), (f) SSG (92.98), (g) SSGL (98.56).
Figure 7. Classification maps and overall classification accuracies (in parentheses) obtained for the AVIRIS Kennedy Space Center dataset using different classification methods. (a) KSRC (89.86), (b) MV (93.30), (c) KJSRC (92.96), (d) CRF (93.49), (e) CKEMAP (97.43), (f) SSG (92.98), (g) SSGL (98.56).
Ijgi 06 00258 g007
Figure 8. OA as a function of β .
Figure 8. OA as a function of β .
Ijgi 06 00258 g008
Figure 9. OA as a function of the number of anchor samples.
Figure 9. OA as a function of the number of anchor samples.
Ijgi 06 00258 g009
Figure 10. OA as a function of the number of training samples for different classification methods when applied to (a) the AVIRIS Indian Pines dataset, (b) the ROSIS University of Pavia data set, and (c) the AVIRIS Kennedy Space Center dataset.
Figure 10. OA as a function of the number of training samples for different classification methods when applied to (a) the AVIRIS Indian Pines dataset, (b) the ROSIS University of Pavia data set, and (c) the AVIRIS Kennedy Space Center dataset.
Ijgi 06 00258 g010
Table 1. Difference between the proposed spatial-spectral graph regularization with location information (SSGL) method and the deep learning methods.
Table 1. Difference between the proposed spatial-spectral graph regularization with location information (SSGL) method and the deep learning methods.
Deep Learning MethodsSSGL
No. of parametersSeveral parameters needed to be trained.Four model parameters: β , γ , λ and α . One algorithm parameter μ .
No. of training samplesAbundant training samples needed.Moderate (or even limited) training samples needed.
Computational costExpensiveModerate
FlexibilityRobust to the spectral distortion.Restricted to the test area.
AccuracyHighModerate
Table 2. Sixteen ground-truth classes in the AVIRIS Indian Pines dataset and the number of training and test pixels used in the experiments.
Table 2. Sixteen ground-truth classes in the AVIRIS Indian Pines dataset and the number of training and test pixels used in the experiments.
ClassSet
No.NameTrainTest
C01Alfalfa351
C02Corn-no till721362
C03Corn-min till42792
C04Corn12222
C05Grass/pasture25472
C06Grass/trees38709
C07Grass/pasture-mowed224
C08Hay-windrowed25464
C09Oats218
C10Soybeans-no till49919
C11Soybeans-min till1242344
C12Soybean-clean till31583
C13Wheat11201
C14Woods651229
C15Bldg-grass-tree drives19361
C16Stone-steel towers590
Total5259841
Table 3. Nine ground-truth classes in the ROSIS University of Pavia dataset and the number of training and test pixels used in the experiments.
Table 3. Nine ground-truth classes in the ROSIS University of Pavia dataset and the number of training and test pixels used in the experiments.
ClassSet
No.NameTrainTest
C1Asphalt406812
C2Meadow4018,646
C3Gravel402167
C4Trees403396
C5Metal sheets401338
C6Bare soil405064
C7Bitumen401316
C8Bricks403838
C9Shadows40986
Total36043,563
Table 4. Thirteen ground-truth classes in the AVIRIS Kennedy Space Center dataset and the number of training and test pixels used in the experiments.
Table 4. Thirteen ground-truth classes in the AVIRIS Kennedy Space Center dataset and the number of training and test pixels used in the experiments.
ClassSet
No.NameTrainTest
C01Scrub39722
C02Willow swamp13230
C03Cabbage palm hammock13243
C04Cabbage palm/oak hammock13239
C05Slash pine9152
C06Oak/broadleaf hammock12217
C07Hardwood swamp699
C08Graminoid marsh22409
C09Spartina marsh26494
C10Cattail marsh21383
C11Salt marsh21398
C12Mud flats26477
C13Water47880
Total2684943
Table 5. Classification accuracies for the AVIRIS Indian Pines dataset using different classification methods. The best results are highlighted in bold. AA: average accuracy; CKEMAP: composite kernel with extended multi-attribute profile features; CRF: condition random fields; KA: kappa coefficient of agreement; KSRC: kernel sparse representation classification; KJSRC: kernel joint SRC; MV: majority voting; OA: overall accuracy; SSG: spatial-spectral graph regularization with location information; SSGL: spatial-spectral graph regularization with location information.
Table 5. Classification accuracies for the AVIRIS Indian Pines dataset using different classification methods. The best results are highlighted in bold. AA: average accuracy; CKEMAP: composite kernel with extended multi-attribute profile features; CRF: condition random fields; KA: kappa coefficient of agreement; KSRC: kernel sparse representation classification; KJSRC: kernel joint SRC; MV: majority voting; OA: overall accuracy; SSG: spatial-spectral graph regularization with location information; SSGL: spatial-spectral graph regularization with location information.
Class TypeKSRCMVKJSRCCRFCKEMAPSSGSSGL
C0156.6772.3581.5780.7889.6168.4391.18
C0278.3385.2787.2288.8491.2479.9095.01
C0364.3182.1592.2989.3196.7476.9896.48
C0452.0790.8687.8891.9884.9169.3785.68
C0589.0390.9790.9788.8892.6590.1593.09
C0696.4696.1196.3399.3198.7098.5898.77
C0774.5819.1719.1756.2596.6788.3397.08
C0898.7599.7899.5799.6699.7099.4499.48
C0957.7830.0040.0020.0097.7851.6793.33
C1072.8786.7485.9186.5990.4884.4192.76
C1182.4397.4293.9597.1196.9293.7897.42
C1276.7497.7595.6197.9991.3690.0396.72
C1398.7610099.0099.6099.5099.4599.55
C1495.3099.7897.9998.5399.0698.4999.29
C1553.8069.5686.2690.6195.5173.1690.42
C1688.5691.8996.7896.3393.7887.8988.67
OA(%)81.3391.9292.3993.8395.1788.9796.16
AA(%)77.2881.8684.4186.3694.6684.3894.68
KA(%)78.6690.7591.3492.9694.5087.3995.62
Time(s)17.0317.3317.3118.4823.4834.6936.16
Table 6. Classification accuracies for the ROSIS University of Pavia dataset using different classification methods. The best results are highlighted in bold.
Table 6. Classification accuracies for the ROSIS University of Pavia dataset using different classification methods. The best results are highlighted in bold.
Class TypeKSRCMVKJSRCCRFCKEMAPSSGSSGL
C173.2591.6992.0191.9297.3093.3097.49
C283.2493.2395.2594.7794.9497.0198.48
C378.6893.0190.9091.5491.8690.5897.70
C491.6586.3989.3291.5895.5894.4494.90
C599.4299.1799.4999.8999.5799.3799.36
C684.7798.2099.2699.5396.8598.7098.90
C792.6599.7095.3099.0698.4099.7099.86
C877.9998.0070.7394.7295.5296.1199.00
C999.2795.8689.6299.7399.6099.5099.51
OA(%)82.9693.8892.3894.8695.8396.2498.19
AA(%)86.7795.0391.3295.8696.6296.5298.36
KA(%)78.2192.0490.0593.3294.5695.0997.63
Time(s)98.80100.61137.69107.55322.54224.29210.24
Table 7. Classification accuracies for the AVIRIS Kennedy Space Center dataset using different classification methods. The best results are highlighted in bold.
Table 7. Classification accuracies for the AVIRIS Kennedy Space Center dataset using different classification methods. The best results are highlighted in bold.
Class TypeKSRCMVKJSRCCRFCKEMAPSSGSSGL
C0195.6910010010099.0399.1899.04
C0285.0979.4380.6188.3088.3990.2295.35
C0390.6297.9497.9498.3197.0898.0298.27
C0446.7863.0550.9256.9086.8645.4888.49
C0561.6482.7682.6382.3087.6380.1392.57
C0645.5873.0469.4560.0589.0835.0299.12
C0783.5492.8310098.9989.2999.09100
C0889.2999.1299.2998.7898.0798.2699.05
C0996.0198.4298.4298.4298.5498.3498.34
C1095.3087.0887.5710099.1999.6399.71
C1194.3086.8390.6598.9497.7496.7197.14
C1288.0510078.0797.1797.0495.3097.63
C1399.84100100100100100100
OA(%)88.4593.0190.7094.3596.6392.1598.01
AA(%)82.4489.2787.3590.6394.4687.3497.29
KA(%)87.1292.1989.6093.6996.2591.2297.78
Time(s)111.68116.40576.78153.00626.01192.24198.11

© 2017 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).
ISPRS Int. J. Geo-Inf. EISSN 2220-9964 Published by MDPI AG, Basel, Switzerland RSS E-Mail Table of Contents Alert
Back to Top